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ABSTRACT 


This thesis examines the issues faced by the Program Manager in providing for 
Post Deployment Software Support (PDSS) of the U S. Army's Special Operations 
Aircraft, MH-60K and MH-47E. PDSS of Department of Defense weapon systems is 
becoming increasingly important for several reasons. First, weapon systems functions are 
migrating from hardware to software. Second, these functions are migrating to software 
because it is flexible. Third, because software is flexible, it continues to evolve throughout 
its lifecycle. Finally, this evolution accounts for approximately 70 percent of the lifecycle 
cost of software. This thesis presents a case study of PDSS of the U.S. Army's Special 
Operations Aircraft. The case analysis identifies significant management issues in the 
following areas: acquisition management, project management, and software engineering. 
Conclusions drawn from the analysis reveal that the success of PDSS is dependent on 
effective program management. Effective management of a software acquisition 
involving PDSS requires the following: planning; a cooperative buyer-seller relationship; 
selection of the proper contract type; technical support resources; and the use of proven 
software management techniques such as metrics and process maturity assessments. 
Implementing the recommendations included in this thesis should improve the future 
management of PDSS. 
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I. INTRODUCTION 


A. GENERAL 

Over the past 30 years, the Department of Defense (DoD) has embraced computer 
technology along with the private sector. As a result, increasingly complex weapon 
systems have been produced. Systems such as the Vietnam era F-4 aircraft, which had 
virtually no computer code, have been replaced over time by systems such as the F-16D 
which has approximately 236,000 lines of computer code. [Ref. 1, p. 30] While software 
may offer the promise of increased system flexibility and capability, and reduced operator 
workload, the software development process that yields these benefits is still inherently 
risky. 

DoD's most recent experience with the complexities and risks in the acquisition of 
Mission Critical Computer Resources (MCCR), weapon system computers and their 
software, has been illustrated in systems such as the Air Force's C-17 cargo aircraft, the 
Navy's F-14D fighter aircraft, and the Army's Fire Direction Data Manager. These and 
numerous other systems have been plagued with problems such as long development 
schedules, ill-defined requirements, the inability to meet performance specifications, and 
cost overruns. From its investigation of DoD's software troubles, the General Accounting 
Office (GAO) concluded in a December 1992 report [Ref. 2], that DoD's software 
problems fall into three general categories: management; requirements definition; and 
testing. 

Why does DoD have these software problems? The Defense Systems 
Management College's Mission Critical Computer Resources Management Guide [Ref. 3, 
pp. 2-6 - 2-8] provides some insight into the causes of DoD's software problems. Among 
these are: 

• Weapon system software has extremely demanding requirements which drive 
system designers toward extremely complex solutions. 

• The requirements for the software are usually not defined until late in the 
development cycle, and then often continue to expand. 
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• Weapon system software with its real-time and fault free operating requirements 
is the most difficult type of software to develop. 

• Despite the complex demands placed on software, development budgets and 
schedules are overly optimistic. 

• There is a lack of experienced software managers within the Government. 

• The supply of skilled software developers cannot meet the demands of both the 
military and the civilian market. 

• Software engineering is still an immature discipline, and is just now beginning to 
develop and apply structured methods in software development. 

Despite these problems, weapon systems software has become a system unto itself. 
It is essentially the "glue" [Ref. 1, p. 32] that cements together and integrates the various 
sub-systems of the weapon system. Because of its central role in modem weapon system 
design, software is now the critical path in system development and the key to meeting 
demanding performance requirements. 

As weapon system software has grown in importance, so have the costs and 
challenges associated with maintaining it. In 1992, the GAO estimated that DoD spends 
from $24 billion to $32 billion on software annually. This accounts for between eight and 
11 percent of the Defense budget. [Ref. 4] Over the system life-cycle, as much as 70 
percent [Ref. 3, p. 7-3] of the cost of software systems is spent on support and 
maintenance. As current and future weapon systems mature, and as the inventory of 
weapon systems software continues to grow, the area of software maintenance (what DoD 
calls Post Deployment Software Support (PDSS)) will become increasingly expensive. 
One of the primary causes of the high cost of software maintenance is that unlike 
hardware, software tends to decay under the impact of the many small changes that users 
request in an effort to enhance a system's functionality. [Ref. 5, p. 147] 

In fact the term "software maintenance" is somewhat misleading. It generally 
includes not only the fixing of software defects, but also includes enhancements which 
meet new user requirements, and improvements in the software's quality. While many 
weapon systems have their PDSS performed by an organic DoD software support activity. 
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some systems depend on the software development contractor for support. One such 
system is the U.S. Army's Special Operations Aircraft (SOA). 

The SOA program was an Acquisition Category (ACAT) II program. It originally 
began as a Non-developmental Item (NDI) acquisition to provide 26 MH-47E and 23 
MH-60K aircraft for the U.S. Amy Special Operations Command. The SOA program 
modified the Boeing CH-47D and the Sikorsky UH-60L helicopter airframes to provide 
Special Operating Forces (SOF) aviation with an enhanced capability to conduct 
clandestine, deep penetration airlift missions, under adverse weather conditions, over all 
types of terrain. [Ref. 6, pp. 2-8 - 2-12] The heart of the system, which is identical on 
both airframes, is the software intensive Integrated Avionics Subsystem (IAS). This 
system, developed by IBM Federal Systems Division (now LORAL Federal Systems) 
under subcontracts with the aircraft manufacturers Boeing and Sikorsky, integrates a total 
software system which consists of approximately 380,000 source lines of computer code. 

The SOA Program Computer Resources Management Plan [Ref. 7] originally 
specified that the PDSS of the fielded IAS would be performed by the Software 
Engineering Directorate at the U.S. Army Communications-Electronics Command 
(CECOM). However, a 1993 Process Action Team (PAT), commissioned by the Product 
Manager, conducted a cost-benefit analysis [Ref. 8] and recommended that PDSS of the 
Special Operations Aircraft should be performed by the IAS manufacturer, LORAL. 

Since that time, the aircraft have been fielded and program management 
responsibility has transitioned from the Product Manager Special Operations Aircraft (PM 
SOA) to the Technology Applications Program Office (TAPO). This office is subordinate 
to the U.S. Army Special Operations Command, and co-located with the Army Aviation 
and Troop Command (ATCOM) in St. Louis. Since July 1994, the PDSS of the IAS has 
been performed by LORAL under an acquisition strategy which includes a one year 
firm-fixed-price contract with two one year options. The Program Manager (PM) is 
dissatisfied with this PDSS arrangement for a number of reasons, chief being the first year 
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cost of $4.45 million, which he feels is too much. Nonetheless, he has elected to keep the 
PDSS effort with LORAL for at least the near term while he learns more about 
management of software acquisition and software support and develops a long range 
acquisition strategy for PDSS. 

B. AREA OF RESEARCH 

Post Deployment Software Support is but one of the many issues involved in DoD 
software acquisition management. This thesis will examine the issues faced by the 
Program Manager of the Special Operations Aircraft for PDSS. The primary objective of 
the thesis is to identify and analyze the significant issues associated with the current PDSS 
program and to recommend actions to address these issues. The secondary objective of 
the thesis is to document the lessons learned from the SO A program's PDSS experience to 
assist other DoD Program Managers in planning and conducting PDSS. 

C. RESEARCH QUESTIONS 

The research questions for this thesis are divided into two categories: primary and 
subsidiary. 

1. Primary 

In light of the Program Manager's PDSS situation, the primary research question 
for this thesis is: What are the significant management issues associated with the current 
SOA Post Deployment Software Support program and what actions can be taken to 
address these issues? 

2. Subsidiary 

The following subsidiary questions support this research: 

• What is PDSS and what are its essential elements? 

• What are the principal policies and guidance concerning PDSS? 

• What are the elements of the current SOA PDSS program? 
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• What are the significant management issues associated with the current PDSS 
program? 

• What actions can be taken to address these issues? 

• What lessons can be learned from the SOA program's experience with PDSS? 

D. SCOPE 

This thesis will be limited to a discussion of the software acquisition management 
issues facing the Program Manager in providing Post Deployment Software Support for 
the Special Operations Aircraft. While there are technical issues involving software 
engineering processes, these will be addressed only to the degree necessary to clarify the 
management situation. 

E. METHODOLOGY 

Analysis of the research questions was accomplished using three techniques. First 
a literature search was conducted for both software maintenance and support, and the 
SOA Program. Information was obtained from: the Defense Logistics Studies Information 
Exchange (DLSIE); the Defense Technological Information Center (DTIC); the Air 
Force's Software Technology Support Center; the Institute of Electrical and Electronics 
Engineers (IEEE) data base and other sources from the Dudley Knox Library at the Naval 
Postgraduate School; PM SOA; TAPO; and the Army's Office of the Director Information 
Systems for Command, Control, Communications, and Computers (ODISC4). Second, 
interviews were conducted with personnel from PM SOA, TAPO, ATCOM, and the 
PDSS contractor, LORAL. Finally, the analysis of the issues identified with the current 
PDSS program was conducted in light of the literature review, interviews, and the 
researcher's knowledge and insight gained from personal involvement with the SOA 
program. The result of this analysis is a set of recommendations to address the PDSS 
program issues. 


5 




F. 


ORGANIZATION 


Chapter II presents the background. It addresses the history of DoD's experience 
with weapon systems software acquisition, explains players and the processes involved in 
the DoD software acquisition environment, examines key elements of software support, 
and reviews the principal policies and the guidance concerning PDSS. 

Chapter III provides the relevant history of the Special Operations Aircraft 
Program, and documents the case description of the current PDSS program. It 
summarizes the key program decisions, provides a description of the aircraft and software 
systems, and explains the key elements of the current PDSS acquisition management 
process. 

Chapter IV presents the analysis of the issues associated with the current PDSS 
program. This analysis will focus on the acquisition management, project management, 
and software engineering issues. 

Chapter V presents the conclusions and the recommendations to address the issues 
identified and analyzed in Chapter IV. It will also provide the answers to the research 
questions and areas for further research. 
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II. BACKGROUND 


A. INTRODUCTION 

This chapter will provide the background of the DoD software acquisition 
management environment and PDSS. The chapter is divided into three sections. The first 
section provides a history of DoD's experience in acquiring weapon systems software. 
The second section will address the DoD software acquisition management environment, 
including the players and their processes, and the policies which guide them. Finally, the 
third section will address software support and PDSS. 

B. SOFTWARE ACQUISITION - THE HISTORICAL CONTEXT 
1. Software Engineering is the Foundation 

The Defense Systems Management College's Mission Critical Computer Resources 
(MCCR) Guide [Ref. 3] points out that the development of computer software as a 
recognized activity is less than 40 years old. The nation's universities did not establish 
separate computer science departments until the late 1960s, and the term "software 
engineering" did not appear until 1968. Therefore, unlike other engineering disciplines, 
software engineers lack any significant body of time-tested practices, procedures, and 
tools normally available in other engineering disciplines. [Ref. 3, p. 2-1] The MCCR 
Guide quotes author Richard Fairley's definition of software engineering as: 

...the technological and managerial discipline concerned with 
systematic production and maintenance of software products that are 
developed and modified on time and within cost estimates. [Ref. 3, p. 5-1] 

It is important to understand that software engineering, the foundation of the 
process by which software is developed, must be scientific and disciplined. Until very 
recently, it has not. While the trend is slowly changing, software engineering is relatively 
still in its infancy, a rather immature discipline involving too much art and craft, and not 
enough structure - an activity which has been called a "black art." [Ref. 1, p. 30] 
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2. Software's Growing Role 

In spite of the fact that software engineering is immature, DoD's appetite for 
complex weapon systems computers and software has grown dramatically over time. In 
his article, Is Software DOD's Achilles' Heel?, [Ref. 1] James Kitfield examines the 
possibility that DoD's demand for high quality weapon systems software may someday 
exceed the nation's capacity to supply it. Kitfield points out that in the software intensive 
aircraft avionics market alone, demand for software increases 25 percent annually. In the 
same period, the education system produces only four percent more software 
programmers. He provides insight into the tremendous leaps made in weapon systems 
software content since the Vietnam War-era with the figures shown in Table 1. 


SYSTEM 

LINES OF SOURCE CODE 

F-4 

Virtually None 

F-16D 

■BHi 

C-17 

625-000 to 750,000 

B-1B 

1,200,000 

ATF 

5,000,000 to 7,000,000 


25,000,000 


Table 1. Weapon Systems Software Growth 


As weapon systems have increased in software content, designers have placed 
increasing requirements on software to perform functions which had been previously 
performed by hardware. In performing these functions, software offers both increased 
system flexibility and reduced operator workload. As shown in Figure 1., the trend calls 
for an ever increasing role for weapon systems software. 
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Figure 1. Software's Growing Influence 
From [Ref. 3, p. 2-3] 


Given its increasing role, software has become a system unto itself, integrating all 
the various functions and sub-systems of the weapon system. But this role is not without 
a degree of risk. Despite the difficulties in engineering complex software, system's 
designers frequently depend heavily on it to solve performance problems. In this regard, 
Kitfield quotes Jack Kramer, director of software engineering for the Institute for Defense 
Analysis: 

Because software is the glue that cements the weapon together, all 
the problems that they {the designers ) were unable to solve elsewhere in 
the system - which are obviously some of the hardest - are pushed onto 
software. 


Kramer goes on to warn that what may seem to a DoD program manager to be a 
trivial software change, may have dramatic consequences. "Program managers wouldn't 
think of adding a second engine to an aircraft at the last minute, but they sometimes try 
similar changes in software." 

3. Software Acquisition Problems and Risks 

Along with software's increasing role have come numerous problems and risks. 
The subject of numerous General Accounting Office (GAO) reports, these problems, and 
their underlying causes were summarized in a 1992 report. [Ref. 2] While the program 
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specific problems differed, the GAO found that they have occurred in three general areas 
as outlined below: 

• Management 

° Lack of management attention/oversight. 

° Lack of adequate software management concepts, methods, practices. 

° Lack of adequate planning. 

° Development proceeded despite serious problems. 

• Requirements Definition 

° Lack of well-defined requirements. 

0 Requirements change to meet new missions. 

° Lack of overall system perspective. 

° System not ready to adapt to change. 

° Software products cannot/may not meet security requirements. 

• Testing 

° Lack of adequate testing methods and approaches. 

° Lack of system-level integration testing. 

The final impact of these problems has usually been a reduced level of system 
performance or the outright inability of the system to meet its requirements. Along with 
these deficiencies also come inevitable cost and schedule overruns. But why have DoD 
software development programs experienced these problems? The same GAO report 
highlights some of the reasons behind these software acquisition difficulties: 

• Mission critical systems require millions of lines of highly complex software. 

• The process to develop high quality software is poorly understood. 

• Unlike hardware, it is difficult to measure software's essential characteristics, 

such as correctness, robustness, performance, and integrity. 

• Developers have difficulty in measuring the progress of development. 

• These systems must operate in a real-time mission environment. 
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Given these problems, it is clear that software development has historically been 
inherently risky and difficult for the DoD Program Manager. But these software 
development problems and risks are not unique to DoD. From his company's study of 
several thousand software projects, both military and civilian. Software Productivity 
Research's founder Capers Jones lists what he has found to be the 10 most serious 
software risks: [Ref 5, p. 47-60] 

• Inaccurate Metrics - Inaccurate measures of productivity such as the 
commonly used "Lines of Code" (LOC) or "Source Lines of Code" (SLOC). 

• Inadequate Measurement - Project cost tracking and cost collection systems 
omit or "leak" major portions of project cost. 

• Excessive Schedule Pressure - A software development schedule set by some 
arbitrary method rather than by a realistic estimate of the effort required to 
complete the project. 

• Management Malpractice - Managers who are not properly trained or skilled 
to meet the challenges of software development. 

• Inaccurate Cost Estimating - Under estimating the cost of the software 
project. 

• Silver Bullet Syndrome - Management's belief that a single tool or method will 
solve all of their software problems. 

• Creeping User Requirements - A constant flow of new and unanticipated 
requirements changes during software development. 

• Low Quality - Significant user dissatisfaction with the delivered product or high 
software defect rates. 

• Low Productivity - Projects which fail to proceed at the rate which 
management expects. 

• Canceled Projects - The "average" canceled project is about a year late and 
approaches or exceeds twice its planned budget. 

The GAO's findings [Ref. 2] certainly indicate that DoD shares many of these 
common software risks with the private sector. However, Jones also provides what he has 
found to be the most common risks in DoD specific software projects: [Ref. 5, p. 33-35] 

• Excessive Paperwork - This risk is present in 90% of all DoD projects. The 
paperwork or documentation traditionally required by Military Standards such as 
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DoD-STD-2167A often accounts for as much as 52% of the total cost of the 
software project. 

• Low Productivity - Present in 85% of DoD Projects. Jones defines this risk as 
projects which are 50% lower in productivity than the U.S. average. 

• Long Schedules - Present in 75% of projects. While the term "long schedule" 
is relative, Jones believes that military projects which last more than five years 
will have troubles adapting to a world which is changing faster than the software 
is developing. 

• Creeping User Requirements - Present in 70% of projects. Jones has found 
that on a typical four to five year project, as much as 50 to 60% of the delivered 
software functionality is in the form of creeping requirements. 

• Unused or Unusable Software - Present in 45% of projects. This risk is tied to 
the long development schedules and the fact that the computer hardware on 
which the software was built to run is outdated by the time the project is 
completed. 

4. Management is the Key 

The problems and risks associated with DoD software development projects have 
given rise to the perception that there is in DoD, and in the software industry in general, a 
"software crisis." But author Robert L. Glass responds to this perception by asking a 
rather probing question: "If there is truly a software crisis, then how could we be sending 
vehicles into space?" He goes on to theorize that the root cause of the "crisis" is not in 
how we build software, but rather in how we think we build software. He believes that 
the problem lies not in the state of software engineering technology, but rather in how we 
manage that technology. [Ref. 9, p. 27] 

This observation concerning management is echoed both by the GAO [Ref. 2], and 
the Defense Science Board Task Force on Military Software, [Ref. 10, p. 7] which stated 
in its 1987 report that: 

In spite of substantial technical development needed in requirements 

setting, metrics and measures, tools, etc., the Task Force is convinced that 

today's major problems with military software development are not 

technical problems, but managerial problems. 
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While much of the criticism in the report is directed toward corporate-level DoD software 
management, the ultimate responsibility for the success or failure of a weapon system 
acquisition involving software lies with the Program Manager. Management at the project 
level is key to successful software development. The next section describes the 
environment in which the Program Manager of a software-intensive system must operate. 

C. THE SOFTWARE ACQUISITION MANAGEMENT ENVIRONMENT 

This section describes the software acquisition management environment. This 
environment is described in terms of its key players, and the processes they use in 
developing weapon systems software. 

1. The Players and Processes 

Marciniak and Reifer [Ref. 11, p. 50] define software acquisition management as, 
"...the process of acquiring custom software, usually by contract from some third party." 
It includes activities such as planning, contracting, budgeting, evaluating and managing 
performance, and providing for the future support of the system. As shown in Figure 2, 
the primary players involved in the software acquisition management process are the 
customer or user, the contracting activity or buyer, and the developer or seller. 

Within the context of the DoD acquisition environment, the user is the activity 
which has generated the Mission Need Statement and the Operational Requirements 
Document for the system, this activity is often referred to as the Combat Developer. The 
buyer is the DoD Program Manager, or the Materiel Developer. Finally, the developer is 
the contractor, or in some cases an organic DoD software development activity. 
Marciniak and Reifer point out that the process of acquiring weapon system software 
requires the interaction of these players in three overlapping processes as shown in Figure 
3. The acquisition management process is primarily the responsibility of the Program 
Manager, the software engineering process belongs to the contractor, while both parties 
share responsibility for project management. 
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Figure 2. The Software Acquisition Management Players 
From [Ref. 11, p. 51] 



Figure 3. The Software Acquisition Management Relationships 
From [Ref. 11, p. 51] 










2. Acquisition Management 

The DoD software acquisition management process is influenced by both policies 
and standards. This section discusses the key policies and standards which impact 
software acquisition. 

a. Acquisition Management Policy 

DoD weapon systems acquisition, including the software component of the 
system is conducted within the policy framework established by DoD Directive 5000.1, 
Defense Acquisition, [Ref. 12] and DoD Instruction 5000.2, Defense Acquisition 
Management Policies and Procedures [Ref. 13], Together, these policies establish a 
disciplined system acquisition life-cycle of discrete phases which include: (1) Phase 0 - 
Concept Exploration and Definition; (2) Phase I - Demonstration and Validation; (3) 
Phase II- Engineering and Manufacturing Development; (4) Phase III - Production and 
Deployment; (5) Phase IV - Operations and Support. At the beginning of each phase 
there is a milestone decision point at which the status of the program is reviewed to 
determine if it should proceed to the next phase. The ultimate objective of this 
management framework is to "translate operational needs into stable, affordable 
programs." [Ref. 12, p. 1-1] The major activities within this process are: [Ref. 12, pp. 1-4 
- 1 - 6 ] 

• Development of acquisition strategies and program plans. 

• Risk Management. 

• Contract type selection. 

• Development of program objectives and baselines. 

• Competition and source selection. 

• Contractor management. 

b. Software Standards 

In addition to DoD Directive 5000.1 and DoD Instruction 5000.2 which 
provide the high-level acquisition guidance, the DoD acquisition environment has also 
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been heavily influenced by specifications and standards. The extent of this influence has 
been reduced by Defense Secretary William Perry's June 1994 memorandum, 
Specifications and Standards - A New Way of Doing Business. [Ref. 14] Because the 
software industry has been slow to develop its own military related standards, the 
following standards have had a great impact, and will continue to impact 
software-intensive programs for the near future: 

• DoD-STD-2167A - Defense System Software Development. [Ref. 15] This was 
DoD's previous attempt to bring discipline to the weapon systems software 
development process. It detailed the requirements for software development, 
testing, configuration management, documentation, and transition of support. It 
has been superseded by MIL-STD-498, Software Development and 
Documentation and is used in ongoing programs only. 

• DoD-STD-2168 - Defense System Software Quality Program. [Ref. 16] 
Established quality control of the software development process and products. 

• MIL-STD-498 - Software Development and Documentation. [Ref. 17] Provides 
a single Standard for all DoD software development, both weapon systems and 
information systems. It is interesting to note that MIL-STD-498 was approved 
after the Defense Secretary's policy on minimizing the use of such standards. 

3. Project Management 

Project management bridges the gap between the acquisition management and 
software engineering processes. Because it is primarily concerned with control of the 
software engineering process, the primary player in this process is the software 
development contractor. However, project management intersects acquisition 
management at the project level, the realm of the DoD Program Manager, and is one of 
his primary responsibilities. 

a. Definition 

Software project management is defined as: 

...a system of procedures, practices, technologies, and know-how 

that provides the planning, organizing, staffing, directing, and controlling 

necessary to successfully manage an engineering project. [Ref. 18, p. 15] 
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b. Project Management Activities 

Some of the key management activities which occur within each of the 
functions included in the definition are: [Ref. 18, p. 17] 

• Planning 

° Set objectives or goals. 

° Develop strategies. 

° Determine courses of action and make decisions. 

° Prepare budgets. 

° Document project plans. 

• Organizing 

° Select and establish organizational structures. 

° Define responsibilities. 

° Establish position qualifications. 

° Document organizational structures. 

• Staffing 

° Fill organizational positions. 

° Educate or train personnel. 

° Provide for general development. 

° Document staffing decisions. 

• Directing 

° Provide leadership. 

° Supervise personnel. 

° Coordinate activities. 

° Facilitate communications. 

° Manage changes. 
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* Controlling 

° Develop standards of performance. 

° Establish monitoring and reporting systems. 

° Measure results. 

° Initiate corrective actions. 

° Document controlling methods. 

Each of these topics is addressed in the Program Manager's project management 
plan. [Ref. 19, p. 124] It should be prepared early in the project and be included as part of 
the solicitation package that is provided to the contractor. The Program Manager's plan 
will provide the basis for the contractor’s management plan which will be documented in a 
similar project management plan or in the software development plan. 

c. Management Indicators 

Of particular concern to DoD Program Managers in software acquisition, is 
how they will manage and assess contractor's performance during contract execution. 
Marciniak and Reifer believe that: 

The key to performance management is providing the buyer and 
seller with visibility into the processes and products of software 
development. Without visibility, both parties are blind. Neither has insight 
into whether real progress is being made. To gain visibility, meaningful 
management reviews need to be held. Configuration management and 
quality assurance systems need to be put into place, along with a 
progressive test and evaluation program. Metrics need to be captured and 
reported to provide management with "hard" evidence or indicators of 
progress. [Ref. 11, p. 71] 

As stated above, in addition to a system of management reviews, and sound 
processes for configuration management, quality, and testing, one way in which to provide 
the Program Manager with insight into the contractor's performance is the use of 
management indicators or metrics. There are numerous software metrics which can be 
collected. However, the use of metrics requires data collection and therefore adds to the 
cost of the effort. Program Managers should, to the maximum extent possible, work 
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within the limits of a contractor's existing processes and define a set of metrics which will 
increase insight into the software development process and pinpoint problem areas. The 
following is a list of possible management indicators recommended by Marciniak and 
Reifer: [Ref. 19, pp. 139-149] 

• Memory Utilization - Measures the utilization of computer memory across the 
project. 

• Problem Reports - Problems are anomalies detected during the software 
development process. This metric measures the open problem report trend and 
the number of problems relative to the total source lines of code. 

• Software Size - This metric provides a simple way of tracking development 
progress by lines of code. 

• Personnel Stability - This indicator tracks the total number of people assigned 
to the project by the contractor. 

• Development Progress - This indicator tracks progress toward key activities in 
the software development process. 

• Incremental Releases - Releases are versions of the software produced at 
various times throughout the development. Each release is produced for a 
specific purpose and incorporates corrections and added capabilities developed 
since the last release. This indicator measures the contents of each release. 

• Test Progress - This metric measures the testing progress based on the test 
scheduled and tests successfully conducted. 

• Documentation - This indicator tracks the progress of documentation 
development against the schedule. 

4. Software Engineering 

Software engineering is the process by which the software contractor develops the 
various software products to be delivered under the contract. 

a. Definition 

Software engineering was defined earlier as: 

...the technological and managerial discipline concerned with 

systematic production and maintenance of software products that are 

developed and modified on time and within cost estimates. [Ref. 3, p. 5-1] 
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b. Software Engineering Goals 

The ultimate goal of the acquisition system is to field a system which meets 
the user's requirements. But because software is flexible, and can usually be modified 
more easily than hardware, those requirements evolve over time. This often results in a 
state of almost constant change for the system. The four goals of a software engineering 
discipline address the evolutionary nature of software systems: [Ref. 20, pp. 5-19 - 5-21] 

• Modifiability - The ability to enhance, upgrade, and perform maintenance on 
software without increasing the complexity of the software's original design. 

• Reliability - A critical determinant of software quality where the cost of failure 
is high as in weapon systems software. It is represented by probabilistic 
measures such as the mean-time-between-failure (MTBF) and 
mean-time-to-repair (MTTR). Reliability must be designed into the software, it 
cannot be achieved by testing after the fact. 

• Efficiency - Efficiency is the highest and best use of critical resources. Of 
critical concern in real-time systems is throughput, which is the speed with 
which data can be processed and moved from one software module to another. 

• Understandability - Understandable software reflects a natural view of the 
world. It is structured in such a way so that it is modifiable, efficient, and 
reliable. 


c. Key Elements of Software Engineering 

As shown in Figure 4, software engineering consists of the methods, tools, 
and procedures used to produce software. Software engineering occurs within the context 
of the contractor's software engineering environment (SEE). The SEE consists of the 
automated tools (either purchased commercially or developed internally) to augment the 
development process. It includes a test environment to perform qualification testing, a 
database that records all software development activities, and a software development 
library that facilitates the orderly development and subsequent support of the fielded 
software. [Ref. 20, p. 3-9] 
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SOFTWARE ENGINEERING ELEMENTS 
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Figure 4. The Key Software Engineering Elements 
From [Ref. 20, p. 5-18] 

d. Software Development Models 

Contractors may employ any number of different software development 
models. [Ref. 20, pp. 3-13 - 3-23] The evolutionary development model involves 
developing an operational product early, and then successively developing more refined 
versions. Incremental development involves developing the software product in groups of 
functional capabilities. This is characterized by a, build-a-little, test-a-little approach to 
deliver an initial subset of the final product. Prototyping allows the user to look at and 
evaluate various alternative software designs. This approach enables the user to determine 
if the software meets its requirements. The spiral model is a risk reduction methodology 
which incorporates the basics of the waterfall model (discussed below), and the 
incremental/evolutionary models. 

While the waterfall model is the oldest of the models and is gradually being 
replaced by the others, it provides a simple model with which to describe the basic 
software development process. The waterfall model is depicted in Figure 5. 
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• Software Requirements Analysis - In this phase the software system 
requirements are analyzed and allocated to the various Computer Software 
Configuration Items (CSCI). These are the major components, or subsystems 
defining the software architecture. At the end of this phase, software 
requirements are detailed enough to allow software design to begin. 

• Design - The purpose of this phase is to map the requirements to the software 
architecture. In preliminary design, requirements are allocated to various 
Computer Software Components (CSC) which are distinct parts of a particular 
CSCI. In detailed design, the decomposition continues to the lowest level, the 
Computer Software Unit (CSU). This is a simple building block that can be 
traced back to the system requirements, coded or programmed by a single 
programmer in a short time, and is separately testable. 

• Code and Unit Test - In this phase, individual CSUs are coded and tested. At 
this stage, testing is informal and usually conducted by the programmer. 
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• Subsystem Test and Integration - The purpose of this phase is to integrate the 
CSUs into increasingly higher levels of the software system until all major CSCs 
are tested. 


• Test and Integration - At this point all CSCs have been integrated into a 
functional CSCI. The CSCI is then tested formally in accordance with the 
procedures agreed to in the contract. The purpose of the testing is to qualify the 
CSCI before the software system is integrated and tested in the actual weapon 
system. 


e. Software Process Maturity 

Contractors vary in the maturity and capability of their software 
development processes. Regardless of the specific development model employed, one of 
the key software engineering principles is that the process should be defined and in 
control. A tool which DoD Program Managers may use to measure the maturity of a 
contractor's software process is the Software Engineering Institute's (SEI) Software 
Process Maturity Model. The Model consists of 5 levels, and development at each level is 
better managed and organized than at lower levels. The levels are depicted in Figure 6. 
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Figure 6. The SEI Software Process Maturity Model 
From [Ref. 21, p. 5] 
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The SEI derived this empirical model from the collective experiences of many 
software professionals. One of the primary purposes of the model is to provide a 
framework for improvement of a contractor’s software process. In this regard, the SEI 
found that the maturity model provides the following benefits: [Ref. 22, p. 93] 

• It reasonably represents the historical phases of evolutionary improvement of 
actual software organizations. 

• The model provides a measure of improvement that is reasonable to achieve 
from the prior level. 

• It suggests interim improvement goals and progress measures. 

• The model makes obvious a set of immediate improvement priorities once an 
organizations maturity level is known. 

The maturity model is used in conjunction with a software process assessment 
method such as the SEI's Software Capability Evaluation (SCE). The purpose of the 
assessment is to determine the state of a contractor's software process. The SCE assesses 
the contractor's capabilities in four areas: (1) organization and resource management; (2) 
software engineering process and management; (3) tools and techniques, and (4) software 
development expertise. The SCE, together with the maturity model, serve two purposes 
for the PM. First, they provide a framework for the improvement of a contractor's 
process. Second, they may be used in the source selection of software developers. In this 
respect, the Air Force considers contractors below Level 3 on the maturity model to be 
high risk. [Ref. 20, p. 7-71] 

This section has described the software acquisition environment in terms of the 
players involved and the three processes in which they operate to manage and develop 
weapon systems software. The next section addresses a particular issue in software 
acquisition management, the support or maintenance of fielded weapon systems software. 
What DoD call Post Deployment Software Support (PDSS). 
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D. SUPPORT OF DOD WEAPON SYSTEMS SOFTWARE 


After a software-intensive weapon system has been developed, tested, and 
deployed, it enters the operations and maintenance phase of the system life-cycle. During 
this phase, it can be expected that the system will undergo numerous changes as the users 
identify deficiencies that need to be corrected, and as they request changes which enhance 
or add capabilities to the system. As shown in Figure 7, this phase of the software 
life-cycle accounts for a significant portion of the total life-cycle cost of the system. 
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Figure 7. Software Life-Cycle Cost 
\ After [Ref. 23, p. 4] 

While this distribution of life-cycle cost makes software appear to be expensive, 

one of its benefits is that it can usually be modified faster and more cheaply than hardware. 

An example is the Army's PATRIOT Missile, which was originally developed to be an 

anti-aircraft weapon, but with modified software was able to also engage tactical ballistic 

missiles. Thus, the flexibility that software brings to modem weapon systems does 

provide a cost effective alternative to developing new systems to meet new or changing 

threats. [Ref. 24, p. 37] The process through which fielded weapon system software is 

maintained, supported, and evolves is called Post Deployment Software Support (PDSS). 

1. Definition 

Post Deployment Software Support is defined as: 

...the sum of all activities required to insure that, during the 
production/deployment phase of a mission critical computer system's life, 
the implemented and fielded software/system continues to support its 
original operational mission and subsequent mission modifications and 
production improvement efforts. [Ref. 3, p. 7-5] 
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Throughout the software literature, the terms software maintenance, software 
support, and software evolution are used interchangeably. Regardless of the term used, in 
general, there are two categories of software maintenance activities: [Ref. 25, p. 9] 

• Corrective Maintenance - This category focuses on fixing defects. Defects 
occur when the system does not perform as intended or as specified in the 
system requirements. This category historically accounts for only about 20 
percent of software maintenance effort. [Ref. 26, p. 45] 

• Software Enhancements - These are changes which are not corrective. There 
are two types of enhancements which together account for 80 percent of 
software maintenance effort. 

°. Adaptive Maintenance - These are enhancements to accommodate changes in 
the operational or hardware environment. These account for 25 percent of 
effort. [Ref. 26, p. 45] 

° Perfective Maintenance - This category of changes improve software 
performance, maintainability, or other attributes. These account for 55 
percent of maintenance effort. [Ref. 26, p. 45] 

2. Software Maintenance Problems 

Software maintenance brings with it a set of unique challenges. At the heart of 
these challenges is the issue of the "maintainability" of the software product that is 
delivered from the software development process. Essentially equivalent in definition to 
"modifiability" which was defined earlier, maintainability is defined simply as the ease with 
which the software can be changed to satisfy user requirements, and the effort required to 
add capabilities, correct deficiencies, or adapt to changing requirements. [Ref. 27, p. 13] 
Building in maintainability must be a consideration throughout software development and 
into the maintenance phase of the life-cycle. In a 1987 paper concerning software 
maintainability, Wilma M. Osborne of the National Bureau of Standards presented a list of 
the primary software maintenance problems which challenge the goal of maintainability. 
[Ref. 27] These problems and their corresponding characteristics are presented in Table 2. 
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PROBLEM 

CHARACTERISTIC 

Software Quality 

- Design Quality 

- Computer Code Quality 

- Programming Languages Used 

- Increasing Inventory 

Environment 

- Growth 

- Evolving/Changing 

- Integrating New Hardware and 
Tools 

Management 

- Change/Configuration 
Management 

- Maintenance Techniques 
-Tools to: 

Restructure/Detect Errors/ 
Measure Code Complexity 

- Enforcing Software Standards 

Users 

- Demanding More Capabilities 

Personnel 

- Lack of Experience 

- Morale Problems 

- View of Maintenance: 
Unchallenging/Unrewarding 


Table 2. Software Maintenance Problems 
After [Ref. 27, p. 14] 

As is the case with software development in general, Osborne finds that while 
there are both technical and managerial problems with software maintenance, 
management of the process is the key issue. 

3. PDSS Policy 

In an a attempt to better manage maintenance of weapon systems software, the 
Army has developed a draft PDSS policy issued by the Director of Information Systems 
for Command, Control, Communication and Computers (DISC4). [Ref. 28] The 
highlights of the policy are listed below: 

• Planning and Preparation - Planning for PDSS must begin at the beginning of 
the system life-cycle, and be documented in the Computer Resources Life-Cycle 
Management Plan (CRLCMP). To ensure that the system will be supportable, 
the assigned organic Software Support Activity (SSA) must complete a 
Software Supportability Assessment at each milestone. 
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• Continued Support - PDSS of the system will continue until the system is 
removed from the inventory or earlier if so determined by the Deputy Chief of 
Staff, Operations (DC SOPS). Cost-benefit analysis will be used to determine 
when software should be upgraded or re-engineered, and to determine the level 
of support. 

• Centralized Priorities and Funding - PDSS is a Department of the Army 
(DA) directed program for prioritization and funding. PDSS funding will be 
controlled centrally by DA based on prioritized input from the Major 
Commands. PDSS actions will be prioritized and categorized as either: 

° Minimum Essential (Band 1) - This band is divided into two bands: Minimum 
Sustainment (Band 1.1) - Those activities required to maintain minimum 
essential capabilities; and "Must Do" Requirements (Band 1.2) - Changes 
required to correct problems which impact mission essential capabilities. 

° "Should Do" Requirements (Band 2) - Those activities which will enhance 
system performance or add capabilities. 

• Government Managed - Government personnel are responsible for monitoring 
and guiding all of the various tasks throughout the system life-cycle. In deciding 
whether PDSS will be performed by a contractor, the Government, or a 
combination of both, the Program Manager will develop a "business case 
analysis" which will be coordinated with the Program Executive Officer (PEO) 
and supporting command. If contractor support is selected, the program's 
assigned SSA must perform a Supportability Assessment. If Government 
support is selected, the SSA and Program Manager must jointly develop a 
transition plan. 

4. PDSS Guidance 

The principal DoD "how to" guidance concerning PDSS is given in Military 
Handbook 347, Mission-Critical Computer Resources (MCCR) Software Support 
(MIL-HDBK 347). [Ref. 25] The MBL-HDBK provides guidance concerning the entire 
PDSS process. This includes both the planning and preparation for PDSS, and how to 
conduct software support. 
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a. Software Support Principles 

The MDL-HDBK provides the following software support principles: [Ref. 

25, pp. 12-13] 

• Software support activities occur throughout the system acquisition process. 

• The evolution of software is a continuum of successive software development 
cycles beginning with initial development and continuing with one or more 
follow-on development cycles. 

• Software should continue to evolve as long as essential requirements or 
cost-effective benefits can be derived through software change. 

• The SSA is the Government's agency responsible for providing PDSS 
requirements to the Program Manager and for cost effective PDSS operations 
following transition. 

• Critical MCCR activities are performed by the Government to ensure 
accountability, control, continuity, and compliance. Critical activities include: 
control and identification of software baselines; defining and evaluating software 
quality requirements; system level testing; and planning and managing PDSS. 

• Direct communication between the user and the SSA is important to help 
identify and isolate software problems, and foster support and cooperation. 

b. Pre-Deployment Software Support 

Recall that one of the primary activities involved in the acquisition 
management process is planning. As such, a program's system of software support must 
be based on sound planning. Pre-Deployment Software support are those activities which 
occur prior to a system's initial fielding. They include: (1) Planning for PDSS; (2) 
Identifying PDSS acquisition requirements; (3) Ensuring software supportability; (4) 
Assuring software quality, and; (5) Development and implementation of a transition plan. 
Each of these activities is addressed briefly below: [Ref. 25, pp. 18 - 24] 

• Plan for PDSS - The Computer Resources Life Cycle Management Plan 
(CRLCMP) is the primary document for computer resource planning and 
management throughout the system life cycle. One of the CRLCMP's primary 
purposes is to document the PDSS concept. The PDSS concept is the basis for 
all PDSS planning and is derived from the overall system Integrated Logistic 
Support (ELS) concept. It describes how and to what extent the software will be 
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supported during deployment. If PDSS planning begins early, the set of feasible 
PDSS options will be larger than if planning is done late. In the absence of a 
clear PDSS concept, decisions are made which limit future choices concerning, 
how the system will be supported. A second purpose of the CRLCMP is to 
document the resources required by the SSA to implement the PDSS concept. 
These resources include, facilities, the software engineering environment, and 
personnel. 

• Identify PDSS Acquisition Requirements - The PDSS acquisition 
requirements are those contractual requirements imposed on the software 
developer which will facilitate PDSS. These requirements may include use of 
specific software engineering tools or environments, technical data requirements, 
quality requirements, involvement of the SSA in software development, and 
transition issues. 

• Ensure Software Supportability - Specifying software support requirements in 
a contract does not ensure software supportability. The SSA has a vested 
interest in ensuring supportability through active involvement in the software 
acquisition process. This should include evaluation of the software product's 
supportability. While somewhat subjective, supportability takes into account the 
following: the characteristics of the software product, its completeness, 
correctness, and quality; the software development environment; and the 
resources required to support the software. In ensuring software supportability, 
the SSA may: participate in technical reviews and audits; participate in formal 
qualification testing; or perform Independent Verification and Validation 
(IV&V) (the process by which the requirements and correctness of the software 
are checked by someone other than the developer). 

• Assure Software Quality - Like supportability, quality is not something that 
can be assured by specification alone. It requires the SSA to be actively 
involved in activities such as authenticating specifications, verifying 
requirements, and evaluating proposed quality plans and records. 

• Develop and Implement the Transition Plan - Software transition includes all 
activities required to transition the software development capabilities from the 
software developer to the SSA. It involves the transfer of computer hardware, 
software, data, and experience. The primary focus of the transition is the 
smooth and effective transfer of the Software Engineering Environment (SEE) 
and the Software Test Environment (STE), to allow the SSA to begin 
supporting the software. Software transition is a major undertaking which 
requires detailed planning. Proper planning is essential, without it, transition is 
inefficient. Dr. Thomas Vollman provides examples of the transition problems 
caused by poor planning. [Ref. 29, pp. 191 - 192] These may include a lack of 
adequately trained staffing at the support activity, the lack of defined procedures 
to keep the software and the process under control and a significant loss of 
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corporate knowledge of the system. In a worst case scenario, the SSA will not 
be able to provide PDSS. Important transition activities that must be addressed 
include: 

° Training. 

° Staffing. 

° Turnover, installation, checkout, and integration of any hardware or software 
acquired from sources other than the software developer. 

° Implementation of required PDSS capabilities. This may include problem 
replication and fault isolation, corrective action, software development, 
documentation, and integration and test. 

° Approval and implementation of software configuration and quality 
management plans. 

° Verification that all transition milestones have been completed and that all 
required resources are available. 

° Integration of all PDSS activities into a cohesive PDSS process. 

° Reporting transfer of software product. 

° Determination that security requirements have been met. 

In summary, half of the software support battle is in the proper, early planning of a 
PDSS concept. If a transition to the Government is contemplated, thorough planning and 
preparation is necessary. 

c. The PDSS Process 

As is the case with software development, software maintenance requires a 
structured approach in order to be effective. Similar structured approaches are outlined in 
both the IEEE Standard for Software Maintenance, IEEE Std 1219-1993 [Ref. 30], and in 
Military Handbook 347, Mission Critical Computer Resources Software Support. The 
general PDSS process is depicted in Figure 8. The following paragraphs provide a brief 
discussion of the five major phases: [Ref. 25, pp. 25-27] 

• Initial Analysis Phase - The input to the PDSS process is a change or problem 
report. These may be generated by the user or by programmers working on the 
system. The SSA receives the problem report and provides information to the 
Configuration Control Board (CCB). This includes a detailed description of the 
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change, the classification of the change, analyses of the management, technical, 
and support impacts of the change, the estimated effort required to implement 
the change, and the risks associated with implementing the change. The CCB 
has the authority to make the decision on a change. 
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Figure 8. The General PDSS Process 
From [Ref. 25, p. 25] 

• Software Development Phase - After the CCB makes the decision to 
implement a change, it can be developed and coded by whatever process is in 
place at the support activity. This phase includes the analysis of the 
requirements; preliminary and detailed design of the change; and the coding and 
testing of the particular software configuration item(s) (CSCI) impacted by the 
change. 

• System Integration and Test Phase - The purpose of the integration and test 
phase is to incrementally build the change and to logically test the developing 
system. During this phase, CSCIs are integrated and tested to facilitate the 
isolation and correction of errors. This phase also includes testing of the 
changed software on the target hardware - the weapon system's computer which 
runs the software. If the test version of the software is approved, the 
configuration identification is updated and the process begins again. 

• Product Logistics Phase - The product logistics phase is the final PDSS phase 
and involves the logistic support required for the new operational software 
version which has been created. This may include user training and site 
installation and checkout. The final product of the PDSS process is a delivery 
package in the hands of the trained user and a configuration that matches the 
approved software version. 
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• Support Operations and Maintenance Phase - Support operations occur 
throughout the PDSS process and include activities such as: computer center 
operations, tactical systems operations, and security. 

5. Software Support Summary 

With so much of the life-cycle cost of weapon systems software concentrated in 
the operations and maintenance phase of the system life-cycle, software maintenance or 
support is clearly an area which deserves thorough planning, and management attention. 
Author and software maintenance expert Thomas M. Pigoski gives the following 
perspective on the importance of software support: [Ref. 31, p. 11] 

• Functions are migrating to software - from hardware to software and from 
manual to software. 

• One reason that functions migrate to software is because it is soft - changeable, 
flexible, modifiable. 

• The sustained, incremental, controlled modification of software (the 
enhancement and adaptation activities of software support) is therefore integral 
to the effectiveness of software. 

• Software support requires high degrees of professionalism, technique, and 
technology. 

In regard to these principles he states: 

Yet these propositions themselves were and are in a state of 
change. The migration of functions to software is an ongoing process 
whose scope and limit we cannot yet see. The softness of software is only 
relative, and poses significant dangers and problems of its own. The 
acceptance of software support as a key to the effectiveness of software is 
a social change that still provokes controversy. And the technical and 
managerial environment for software support is something that researchers, 
educators, and vendors are still developing. [Ref. 31, p. 11] 

E, CHAPTER SUMMARY 

Software acquisition has historically been and continues to be a challenging activity 
for the DoD Program Manager. The software acquisition process is a complex interaction 
of three interrelated processes: the acquisition management process, the project 
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management process, and the software engineering process. The product of the 
acquisition process is weapon systems software. Because it is modifiable, it continues to 
evolve throughout the life of the weapon system to meet new user requirements and new 
threats. Software maintenance accounts for about 70 percent of the life-cycle cost of 
software, and more and more weapon systems software is being acquired. Post 
Deployment Software Support is a vitally important part of the software acquisition 
management process. 
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III. CASE DESCRIPTION: POST DEPLOYMENT SOFTWARE 
SUPPORT OF THE SPECIAL OPERATIONS AIRCRAFT 


The President has ordered the cancellation of an operation in 
Iran which was under way to prepare for a rescue of our hostages. The 
mission was terminated because of equipment failure. During the 
subsequent withdrawal of American personnel', there was a collision 
between our aircraft on the ground at a remote desert location in Iran. 
There were no military hostilities, but the President deeply regrets that 
eight American crew members of the two aircraft were killed and others 
were injured in the accident...The nation is deeply grateful to the brave 
men who were preparing to rescue the hostages.... 

White House Statement, 1a.m., April 25,1980 


A. INTRODUCTION 

This official account of the failed Iranian hostage rescue mission, along with the 
images of the charred remains it left behind at "Desert One," dramatically highlighted the 
need for military aircraft which are specially designed and built to withstand the rigorous 
demands of the Special Operations Forces (SOF) aviation mission environment. This 
environment is typified by the conditions encountered during the failed mission. It 
requires the capability for SOF aviation to conduct clandestine airlift missions under 
conditions of limited visibility, any type of weather, and over extremely long-ranges. The 
Army's Special Operations Aircraft (SOA) is part of DoD's solution to the SOF aviation 
challenge. 

This chapter provides the background of the U.S. Army Special Operations 
Aircraft (SOA) Program and a case description of the elements of the SOA Program's 
current Post Deployment Software Support (PDSS) acquisition process. The chapter is 
divided into two major sections. The first section provides the SOA Program background, 
to include a summary of the mission need, the program acquisition strategy, and a 
summary of the Integrated Avionics Subsystem (IAS) software development. The second 
major section provides the description of the SOA Post Deployment Software Support 
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case. It provides a description of the software system being supported by PDSS, the 
original PDSS plan and how the plan was modified by the SOA PDSS Process Action 
Team (PAT). It also includes a discussion of the key players involved in the PDSS effort, 
a description of the PDSS acquisition process, and a summary of the activities 
accomplished to date under the base year, and first option years of the PDSS contract. 

B. SOA PROGRAM BACKGROUND 

1. The Mission Need 

The Special Operations Forces Helicopter Modification (SOF MOD) of the 
UH-60L and the CH-47D aircraft to the MH-60K and MH-47E configurations was 
initiated by the Department of the Army in April 1986. It was in response to the DoD 
Special Operations Forces Airlift Report and the Special Operations Forces Required 
Operational Capability (ROC), both of which are classified SECRET. The SOF MOD 
Project called for the design, integration, modification, and qualification of a Mission 
Equipment Package (MEP). This included an Integrated Avionics Subsystem (IAS) to be 
integrated into both the UH-60L BLACK HAWK, and the CH-47D CHINOOK airframes. 
The purpose of the MEP is to enhance an aircraft's capability to operate reliably in the 
demanding SOF mission environment and to reduce pilot workload and stress. The SOF 
MOD project used a Non-developmental Item (NDI) acquisition approach and was 
originally designated a nonmajor system. The Project Management Office (PMO) was 
initially established simply as a matrix organization within the engineering directorate at 
the U.S. Army's Aviation Systems Command (AVSCOM). 

As indicated earlier, the project was the direct result of the deficiencies in the 
current fleet of standard configuration aircraft to meet the needs of the SOF aviation 
mission. SOF aviation doctrine calls for aircraft which are capable of conducting 
clandestine, deep penetration, low-altitude airlift missions, over all types of terrain, under 
adverse weather conditions, with little or no visibility. Additionally, these aircraft must 
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have increased protection from threat anti-aircraft weapon systems, as well as increased 
self-deployment capability. To meet these requirements the SOA MEP includes: [Ref. 32] 

• An IAS to enhance communications and navigation; 

• Improved Aircraft Survivability Equipment (ASE); 

• Increased armament which includes suppressive weapons; 

• The addition of external and internal fuel tanks, and air-to-air refueling probes; 

• Upgraded engines; and 

• An upgraded transmission on the MH-60K aircraft. 

The configurations of both aircraft are shown in Figures 9 and 10. 

2. SOA Program Acquisition Strategy 

The Product Manager, Special Operations Aircraft (PM SOA) was officially 
established and chartered in September 1987. As an NDI acquisition based on proven 
airframes and avionics equipment, the acquisition strategy did not include a Demonstration 
and Validation Phase. As shown in Figure 11, the SOA Master Program Schedule, a 
Milestone II In-Process Review (IPR) was conducted in October 1987 which authorized 
the program to proceed with development of one prototype aircraft for each airframe. 
The program acquisition strategy included the award of letter contracts for prototype 
research and development in January 1988. The Airframe Contractors (ACs) were Boeing 
for the MH-47E and Sikorsky for the MH-60K Because cost control was a major 
priority, [Ref. 32] these contracts were later definitized as firm-fixed-price contracts. The 
IAS contractor, IBM Federal Systems Division (now LORAL Federal Systems) was 
selected by a Government competitive Source Selection Evaluation Board in August 
1987. However, the Government did not contract directly with IBM. Rather, it became a 
directed subcontractor to the ACs. The modifications developed by the ACs, who had 
total system performance responsibility for integration of the MEP into the respective 
airframes, were planned to be incorporated into the standard aircraft by an Engineering 
Change Proposal (ECP). 
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Figure 9. The MH-60K Configuration 
From [Ref. 33, p. 14] 



Figure 10. The MH-47E Configuration 
From [Ref. 33, p. 13] 






















































DoD Instruction 5000.2 defines Non-developmental items as those which do not 
require development. This includes those items which require only minor modification to 
meet the agency's need. [Ref. 13, p. 6-L-l] In this respect, SO A was considered an NDI 
acquisition because the airframes and each of the avionics components to be integrated 
into them had been previously qualified. Despite the SOA program's NDI status, it was 
not without its risks. The primary source of program risk was that it required 
considerably more "development" than the NDI status had anticipated. However, the 
Army considered the program risks to be manageable. A June 1990 letter from the 
Secretary of the Army to Congress outlined the major areas of risk involved in the SOA 
program. These included: [Ref. 34] 

• The processing capacity of the IAS Mission Processor, AP-102. This was 
considered a medium cost risk. The worst case scenario judged that the 
processor would operate at 93 percent capacity versus the 80 percent goal. 

• Optimization of the flight algorithm for the Multimode Radar, AN/ALQ 174 
which provides the aircraft's terrain following/terrain avoidance capability. This 
was judged to be a low risk item, but the Government would be at risk to 
conduct additional development if optimization could not be achieved within the 
three months of contractor flight testing allocated for the effort. 

• Integration of Government Furnished Equipment. This was considered a low to 
medium cost risk. 

• Retrofit of four production aircraft as a result of concurrent prototype 
development and production. This was considered a low to medium cost risk. 

While mentioned as a low to medium cost risk, in hindsight, the concurrency of 
development and production was in fact one of the major sources of risk in the SOA 
acquisition strategy. This was a characteristic for which the program was criticized by the 
GAO in a 1990 audit report. [Ref. 33] The basis of the criticism was that despite the fact 
that the modification program required the integration of about 40 items (these included 
communications, ASE, and navigation equipment) the development of the prototype 
aircraft was not completed before the Army approved a Low Rate Initial Production 
(LRIP) decision at Milestone III A, in February 1990. The LRIP decision authorized the 
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production of 11 aircraft of each type. The DoD response to this criticism [Ref. 35] 
centered on these facts: 

• That because SOA was not a major acquisition program, regulations did not 
require the program to undergo formal Initial Operational Test and Evaluation 
(IOT&E) prior to a production decision. 

• Operational testing in the program consisted of Early User Test and Evaluation 
(EUT&E) conducted by user pilots and mechanics in conjunction with 
contractor prototype tesing. 

• The SOA testing program was in consonance with NDI acquisition regulations. 

• Each of the avionics components to be integrated into the IAS had been 
previously qualified, or were currently in use on other DoD systems. 

However, at least some of the GAO's concurrency cautions proved true. Later 
cost growth elevated the SOA program from non major program status to a Category II 
Major Program in January 1992. [Ref 6] Growth in the LAP software required an 
upgrade in the Mission Processor to the AP-102A configuration. And in December 1993, 
as one of the program's contracting officers, the researcher personally awarded a contract 
directly to IBM to complete development and qualification of the terrain following (TF) 
capability of the Multimode radar (MMR). [Ref. 6, pp. 2-12 - 2-13] Despite these and 
other challenges encountered during the highly concurrent program, the production 
quantity of 22, MH-60K and 25, MH-47E aircraft have been delivered. The user, the 
160th Special Operations Regiment (Airborne) now has the capabilities originally 
identified in the aftermath of "Desert One." By Fiscal Year 1994, these capabilities cost a 
total of $1.38 billion. [Ref. 6] The development and production schedule for the program 
spanned only eight years. Support of both aircraft is provided by a Life-Cycle Contractor 
Support (LCCS) Joint Venture of both ACs and named Boeing-Sikorsky Air Services. 

3. The IAS Software Development 

For one who has been familiarized with the difficulties involved with weapon 
systems software development, it is not unusual that development and integration of the 
IAS and its software proved to be one of the greatest challenges involved in the program. 
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Over a four year software development, 12 iterations of the Integrated Avionics Program 
(IAP), (the "brain" of the IAS) were built before the system performed in accordance with 
the Prime Item Development Specification (PIDS). In reviewing the history of the 
program, several factors contributed to this problem. First was the cancellation of the 
V-22 Program in November 1989. The SOA program was to have benefited from the 
V-22 as the lead program in the IAS development. A second factor was the disagreement 
between the ACs, IBM, and the Government over exactly what the PIDS required. This 
was especially questioned because both the development and production contracts were 
firm-fixed-price. A third factor concerned a lack of communication between the 
Government and the SOA contractors. 

In an April 1993 interview, the SOA Product Manager (Lieutenant Colonel 
Michael W. Rogers) described the Program's software problem as follows: [Ref. 36, p. 40] 

We were told not to worry about the software, that the next 
iteration of software would solve all the problems we had been 
experiencing in the past. What we had been told was incorrect. The main 
problem we faced with the two helicopter programs was that there was 
little communication between the Government and the contractors. We 
had no way to resolve the software trouble reports (STRs). There was no 
process of keeping track of the software problems we were having. 

To address the software issue. Lieutenant Colonel Rogers formed what he called. 
Team SOA. Team SOA was a Government-contractor team which consisted of the 
following elements: 

• An executive steering group whose membership included Army leadership at the 
general officer level and the contractor CEOs; 

• A management working group consisting of the Government and contractor 
PMs; and 

• Several process action teams which addressed specific software problems. 

Also instrumental to the resolution of the STR problem was user involvement in 
the software development process. This ensured that STRs were resolved to the user 
pilot's satisfaction. Figure 12 depicts the STR resolution trend experienced throughout 
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software development. This trend evidences the complexity of the IAS software 
development. It shows that by the time production software version 10.3.2 was delivered, 
nearly 5,000 STRs had been identified. It also evidences the effectiveness of the 
corrective action, since the number of remaining STRs was later reduced to about 62. 
[Ref. 37] 



From [Ref. 38] 

This section has provided a brief description of the background of the SOA 
program. It highlighted the risks encountered during development and the complexity of 
the IAS software development. The next section describes how the IAS software is 
supported. 



























C. SOA PDSS CASE DESCRIPTION 

1. The Supported Software System 

This section describes the major software-intensive components of the SOA 
system. These include the IAS, non-IAS components such as the MMR, and training 
devices. 

a. The SOA Integrated Avionics Subsystem 

The heart of the capability of the MH-60K and MH-47E Special 
Operations Aircraft, and the focus of the Post Deployment Software Support (PDSS) of 
the aircraft, is the Integrated Avionics Subsystem (IAS). The IAS hardware and software 
are common to both the MH-60K and the MH-47E aircraft. Their primaiy purpose is to 
enhance mission management and reduce crew workload. The IAS consists of 15 Line 
Replaceable Units (LRU) or "Black Boxes." These contain approximately 380,000 SLOC 
in nine CSCIs, which run on ten separate computer processors. The IAS and the other 
non-IAS mission aids are integrated on the aircraft using two, dual-redundant 
MIL-STD-1553B data buses. 

The IAS performs four major functions. These are: [Ref. 39] 

• Navigation - maintaining the aircraft's current state; 

• Guidance - determining the reference path to the next point in the mission plan; 

• Flight Director - providing steering cues to the reference path; and 

• Mission Management - planning the entire desired mission. 

A representation of the IAS is given in Figure 13. 
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The IAS is divided into two major areas, the Mission Management System and 
Mission Aids. It consists of the following LRUs in the quantity indicated: [Refs. 6, 7, 39, 
40] 

• Mission Management System (MMS) 

° Mission Processor, AP-102A Computer (MP) (2) 

° Display Processor (DP) (2) 

° Color Multifunction Display (CMFD) (2) 

° Monochromatic Multifunction Display (MMFD) (2) 

° Control Display Units (CDU) (2) 

0 Remote Terminal Units (RTU) (2) 

• Mission Aids 

° Map Display Generator (MDG)/Data Transfer System (DTS) (1) 

° Aviator's Night Vision Imaging Systems (ANVIS) Display System (1) 

° Grip Assembly, Tracking Handle (1-MH-60K, 2-MH-47E) 

b. Non-IAS Mission Aids 

In addition to the IAS, the following non-IAS mission aids are also 
integrated over the 1553B data bus: [Ref. 41] 

• Multimode Radar - Provides the capability for low-level, terrain following (TF) 
flight in limited visibility. 

• Forward Looking Infrared (FLIR) Sensor - Provides magnified night vision 
capability. 

• Integrated Navigation System - Provides a broad spectrum of sophisticated 
sensors which allow highly accurate navigation to and from the mission area. 
The system includes a Global Positioning System (GPS), Inertial Navigation 
Unit (INU), Doppler Navigation Unit, and a Personnel Location System. 

• Aircraft Survivability Equipment (ASE) - Provides early warning and 
countermeasures against threat anti-aircraft weapons. 

• Communications Subsystem - Provides secure communications, and command 
and control capability with a broad spectrum of radio systems including 
SATCOM, HF, and SINCGARS. 
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• Flight Data Recorder and Video Recorder - Allows crews to record and review 
data transmitted over the 1553B data bus, and the FLIR sensor. 

c. IAS Software 

The IAS software is resident in nine Computer Software Configuration 
Items (CSCIs). The reader should note that the two CSCIs resident in the mission 
processor (MP) are LORAL products, while the remaining CSCIs were developed by 
subcontractors. These are shown in Figure 14 and an explanation is provided in the 
paragraphs below: [Ref. 7, pp. 3-7] 

• Integrated Avionics Program (IAP) CSCI - The IAP provides the overall system 
control and is, in essence, the "brain" of the aircraft. It interfaces and integrates 
information from the more than 35 other subsystems on the aircraft. These 
include: engines, transmissions, MMR, FLIR, communications, navigation, and 
ASE. It is the first of two CSCIs which run in the mission processor (MP). 

• Start-Up Read Only Memory Program (SROMP) CSCI - The SROMP provides 
the ability for the MP to be loaded over the 1553B data bus. Since the IAP is 
identical on both airframes, the SROMP checks the aircraft wiring during start 
up and identifies to the IAP and MP which type of aircraft it is on. The SROMP 
executes on the MP but is stored as firmware, a combination of hardware and 
software. 

• Display Processor (DP) CSCI - Each DP drives two Multifunction displays. The 
DP controls video inputs, construction and display of symbology, and 
timing/interface with the 1553B data bus. It is partitioned into two functions: 

° Display Management Program (DMP). 

° Symbol Generator Program (SGP). 

• Display Processor Memory Loader (DP ML) CSCI - Provides memoiy 
load/verification capability over the 1553B data bus and performs the Power-On 
Built-In-Test (PBIT) for the DP. 

• Remote Terminal Unit (RTU) Operational Flight Program (OFP) CSCI - The 
Remote Terminal Unit provides the interface conversion between the MP and 
the non-1553B data bus compatible avionics components. 

• Remote Terminal Unit Memory Loader (RTU ML) CSCI - This CSCI provides 
memory loader/verification capability over the 1553B data bus and performs the 
PBIT function for the RTU. 
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• Map Display Generator (MDG) CSCI - The MDG CSCI provides the advanced 
map display capabilities. It uses a modified Defense Mapping Agency (DMA) 
Digital Landmass database, aeronautical chart data, and digitized data to 
produce a map display which can be overlaid with mission planning data. It 
consists of two major functions: 

° Map Management Processor (MMP). 

° De-Compression Engine (DCE). 

• Map Display Generator Memory Loader (MDG ML) CSCI - This CSCI 
provides a memory load/verification capability over the 1553B data bus and 
performs the PBIT function for the MDG. 

• Control Display Unit (CDU) CSCI - The CDU provides the pilot/hardware 
interface. Using the CDU keyboard, the crew are able to modify the IAS data or 
update mission related information such as navigation waypoints or 
communications frequency changes. 

• Multifunction Display (MFD) - While not considered a CSCI, the firmware 
resident in each MFD responds to the output generated by its corresponding DP. 
The 24 push buttons around the MFD bezel provide the pilot control interface 
with the IAS. 

d. Training Devices and Support Equipment 

The Special Operations Aircraft system also includes the following 
software intensive training devices and support equipment. They must also be included 
when considering PDSS: [Refs. 6, 39] 

• Desk Top Trainer (DTT) - Trains pilots and maintainers in the use of the SO A 
Mission Management system, and operation and control of the Integrated 
Avionics Program (IAP) through the multifunction display. It runs a CSCI of 24 
thousand SLOC in a desktop personal computer. 

• Part Task Trainer (PTT) - Trains pilots and maintainers through interactive 
simulation of the MH-60K and MH-47E equipment using "Touch Screen" 
technology. It runs two CSCIs of 54 thousand SLOC in the earlier version of 
the mission processor, the AP-102. 

• Combat Mission Simulator (CMS) - A full motion simulator for each aircraft 
located with the user at Fort Campbell. It was developed and is supported by 
the U.S. Army Simulation, Training, and Instrumentation Command 
(STRICOM). 
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• Ground Support System Software Load Device (GSS SLD) - An IBM ThinkPad 
laptop computer which connects to the 1553B data bus to gather maintenance 
data and to load software into IAS processors. 

2. PDSS Plan 

The reader will recall from Chapter II. that one of the keys to effective PDSS is 
prior planning. Such planning is properly conducted during what MIL-HDBK-347 calls 
Pre-Deployment Software Support. It includes activities such as: (1) planning for PDSS; 
(2) identifying PDSS acquisition requirements; (3) ensuring software supportability; (4) 
assuring software quality, and; (5) development and implementation of a transition plan. 
[Ref. 25, pp. 18-24] Each of these activities should be addressed in the weapon system's 
Computer Resources Life Cycle Management Plan. This document is developed by the 
PM, in coordination with the Government Software Support Activity (SSA) that is 
designated to perform PDSS for the system after fielding. 

Throughout the development of the SOA IAS software, the assigned SSA was the 
Software Engineering Directorate, Avionics Branch, at the US. Army 
Communications-Electronics Command (CECOM). CECOM software engineers were 
most closely involved in the software development effort through the conduct of 
Independent Verification and Validation (IV&V) of the software design documentation, 
and code listings. In preparation for the transition of the software engineering 
environment (SEE) from LORAL to CECOM, one of the system's major components, the 
Army SOA Helicopter Avionics System Simulator (AHASS) was re-hosted from a 
mainframe computer to a smaller, workstation system. [Ref. 42] The SSA's plan for 
software support, and a plan to transition software development responsibilities from 
LORAL to the SSA were included in the SOA Computer Resources Management Plan 
(CRMP). The highlights of this plan were: [Ref. 7] 

• The purchase of a complete Software Engineering Environment (SEE) to 
maintain all of the IAS CSCIs. The plan called for 18 months to procure and 
install the SEE. 
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• The conduct of PDSS training through participation in software development 
activities at LORAL's facility, followed by a period of several years during which 
LORAL's involvement in PDSS would be gradually phased out. 

• Transition of PDSS responsibility from LORAL to the SSA 120 days prior to 
the fielding of the last production aircraft. 

• Process flow charts which addressed: the flow of STRs; the configuration 
management process; and the Engineering Change Proposal Process (ECP). 

• Annual software releases for the IAP during PDSS. 

It should be noted, that while the typical Government SSA is Government 
managed, the majority of the software development work is conducted by support 
contractor personnel. This is true not only at CECOM, but is a DoD wide challenge and 
was witnessed by the author at two other DoD SSAs. [Refs. 43, 44] The CECOM 
estimates for man-years (MY) of PDSS labor, and PDSS funding in millions of dollars are 
shown in Tables 3 and 4 respectively. 
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Table 3. PDSS Labor Estimate 
From [Ref. 7, p. 71] 
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Table 4. CECOM PDSS Funding Request 
From [Ref. 45] 
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3. 


PDSS Process Action Team 


Apparently unsure of the PDSS plan called for in the CRMP, the SOA Product 
Manager chartered a Process Action Team (PAT) in January 1993 to evaluate the various 
PDSS options available to the program, and to make a recommendation as to how to 
conduct PDSS "...in a manner that is cost effective and affordable." [Ref. 8, p. 2] 

The PAT consisted of representatives from: the Product Manager's Office; the 
user; the contractors (IBM, Sikorsky, and Boeing); CECOM; and the Aviation and 
Troop Command (ATCOM). It initially considered nine PDSS options. As the PAT met 
over the period from Januaiy to May 1993, the field of PDSS options was narrowed to 
these four alternatives: [Refs. 8, 45] 

• Alternative One - CECOM manages PDSS with its own SEE and contracting for 
IBM support as required. 

• Alternative Two - The SOA PMO manages PDSS through the airframe 
contractors which would subcontract the effort to IBM. 

• Alternative Three - The U S. Air Force manages PDSS in manner similar to 
Alternative One. 

• Alternative Four - The SOA PMO, and then the Technology Applications 
Program Office (TAPO) manages PDSS directly with IBM. 

These four options were then compared in a cost benefit analysis which was 
documented in an Economic Analysis report. [Ref. 8] The analysis revealed that while 
Alternative Four was the second most expensive option, considering a 20 year discounted 
life cycle cost estimate, the benefits of this alternative outweighed the cost. The benefits 
considered for each alternative included: [Ref. 8, pp. 13 - 14] 

• Maintenance Experience - The amount of maintenance experience in the 
organization. The maintenance task differs from that of development, and prior 
experience in maintenance is a benefit. 

• SOA Experience - SOA experience consists of "domain" knowledge of the SOA 
software. Prior or current SOA experience is a benefit. 

• No Transition - Transition is a description identifying the shift from the original 
software developer/maintainer. A loss of knowledge and familiarity, as well as 
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domain expertise, is to be expected during this shift. A lack of a transition is a 
benefit. 

• Short Response Time - The time required for a software change to be 
implemented once the requirements have been determined. This is greatly 
impacted by the number of organizations involved. A short response time is a 
benefit. 

• Facility - Facility is a measure of the availability of assets used to maintain the 
SOA software. A shared facility will impose a cost to timely and accurate 
software maintenance activities. A dedicated facility is a benefit. 

• Translation of Requirements - The translation of requirements through multiple 
layers of communication is a concern. Fewer organizational layers is a benefit. 

• Aircraft Knowledge - This attribute captures the amount of knowledge of the 
total aircraft system. This includes experience with the interaction and 
dependencies involved with the airframe and components, propulsion, pilot, and 
flight envelopes. Configuration management is also considered in this attribute. 

• Other - These factors included the number of contracts involved, and the 
political considerations and possible lack of priority if the effort were performed 
by another service. 

As a result of the PAT's recommendation, and the cost benefit analysis, the 
Product Manager elected to contract with the software developer, LORAL for the first 
three years of PDSS. The PM viewed this alternative as the "Best Value" approach to 
PDSS for the program. [Ref. 38] While this was not, at the time, viewed as the permanent 
direction for support of the software over the entire system life cycle, [Ref. 46] the future 
direction for PDSS beyond the first three years was not addressed by the PAT. On 25 
July 1994, LORAL was awarded a one year firm-fixed-price contract, with two, one year 
options for PDSS of the Special Operations Aircraft. This contract, awarded initially as an 
undefinitized contract action, the base year of which purchased IAP Release 12.0, was 
definitized on 25 April 1995 at a price of $4.45 million, exactly the amount available for 
the effort. [Ref 47] 
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4. 


The Key PDSS Players 


The reader will recall from Chapter II that software acquisition management 
involves the interaction the key players - the user, the buyer, and the software developer 
in the acquisition process. This section identifies the key players involved in the SOA 
PDSS acquisition. 

a. Technology Applications Program Office (TAPO) 

On 1 April 1995, with the Product Manager having fulfilled his charter, the 
Project Management Office, Special Operations Aircraft (PM SOA) was disbanded and 
Program Management (PM) responsibility for the aircraft was transitioned to TAPO. 
TAPO serves as the "Materiel Developer" for the SOA and the other aircraft assigned to 
the 160th Special Operations Aviation Regiment (Airborne) (SOAR). It is under 
operational control of the U.S. Army Special Operations Integration Command and 
located in St. Louis with the Army Aviation and Troop Command (ATCOM). ATCOM 
provides TAPO's matrix support. This organization is shown in Figure 15. 

TAPO's mission is to support the 160th SOAR with the streamlined acquisition of 
required aviation mission equipment. This mission includes: [Ref. 48] 

• The performance of research and development as required. 

• The use rapid, streamlined acquisition procedures to procure NDI mission 
equipment, aircraft systems, and aircraft 

• Management SOF peculiar aircraft modifications, airworthiness release 
procedures, and configuration control. 

• Providing for logistics sustainment of these aircraft systems. 

• To be responsive to the needs of the user. 

• When possible and practicable, transition SOF peculiar equipment to the 
conventional Army aircraft fleet. 

TAPO is headed by a lieutenant colonel program manager. However, the day to 
day program management responsibility for the SOA is with one of the office's project 
officers. At present the SOA Project Officer (henceforth referred to as the PM), is an 
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Army aviator with experience as a SOA test pilot. In addition to the SOA PDSS effort, he 
is also responsible for other projects. Chief among them is responsibility for the ongoing 
effort to develop and qualify the terrain following (TF) capability of the MMR on the 
aircraft. 



Figure 15. Special Operations Aircraft Organizations 
After [Ref. 48] 


TAPO is sparsely manned, and lacks organic systems or software engineers to 


assist the PM in the management of PDSS. However, a software engineer has been hired 
on a support contract through the ATCOM Directorate for Lifecycle Software 
Engineering (LCSE). He assists the PM in the preparation of statements of work and 
technical reviews of LORAL's proposals. Additionally, the PM has provided some funds 
to the Software Engineering Directorate at the U.S. Missile Command (MICOM), 
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Redstone Arsenal, Alabama, and has been assisted for several months by three of 
MICOM's software engineers. The primary purpose of their assistance is to help the PM 
to be a "smart buyer" in the PDSS contract with LORAL. 

There are numerous PDSS issues which concern the PM. These issues include: 
[Ref. 49] 

• The apparent high cost of $4.45 million for the base year of PDSS which 
delivered IAP Source Release 12.0; 

• Identifying the content/functionality of the changes to be incorporated in the 
next software release; 

• Management of the future growth and configuration of the IAS; 

• Developing a capability to conduct impact assessments and cost estimates for 
software changes; and 

• Negotiating with LORAL to obtain all of the desired functionality for a given 
software release, within the allocated budget. 

The bulk of the PMs time spent on the PDSS program is spent in the management 
of the ongoing issues and problems, and fighting the current "brush fires." However, in 
spite of an environment which allows little time for planning, the PM's vision in regard to 
the SOA PDSS program is: [Ref. 50] 

• To be a "smart buyer" who can effectively manage the current PDSS contract to 
ensure the efficient, cost effective delivery of software support services. 

• To identify the future direction for PDSS of the SOA. 

b. The Systems Integration Management Office (SIMO) 

SIMO represents the user, the 160th SOAR and serves as the combat 
developer in the acquisition management process. In this role, it maintains a data base of 
the Government Trouble Reports (GTRs) written against the SOA software. From this 
data base, SIMO selects GTRs to define the PDSS requirement and the content of the 
yearly software release. As one would reasonably expect, SIMO's primary goal in regard 
to PDSS is maintaining a software product that continues to meet the needs of the user. 
However, another typical user goal, which was discussed in Chapter II, is the continued 
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enhancement of the SOA system's capabilities, and this at a minimum cost. Based on 
previous experience with the other user aircraft systems, it is reasonable to expect that the 
user will continue to request enhancement of the SOA IAS. While generally pleased with 
the quality and function of the software, SIMO representatives have mentioned "going 
competitive" after the current contract. These comments were based primarily on the 
perceived high cost of PDSS. 

c. LORAL Federal Systems Company 

LORAL, as the former IBM Federal Systems Division, was the Army 
directed subcontractor which developed the SOA IAS and its software. It is the lead 
contractor in the development, integration and qualification of the TF capability of the 
MMR. In addition, as the PDSS contractor, the company is adapting to a relatively new 
relationship, working directly with the Army as the lead contractor on the SOA program. 
With a well staffed program office and organizations which support systems and software 
engineering, the company prides itself on its performance as the SOA systems integrator, 
and on the quality of the SOA software. As the system integrator, LORAL's goals are: 
[Ref. 39, pp. 15-16] 

• A SOA system that is safe to fly. 

• System configuration control to assure safe/reliable operation. 

• Being a cost effective solution to the SOA program's needs with: 

° Facilities and trained personnel in place. 

° Extensive use of modeling and simulation for efficient system and software 
development. 

° Cost effective and efficient management of subcontractors. 

LORAL's overall goal is a satisfied customer. It plans on staying in this business 
area and believes that it has the skills and resources to provide a "complete solution" for 
the SOA program. LORAL believes that the key to the future of this complex program is 
a closer, "team" relationship among the user, the PM, and LORAL. It believes that this 
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type of relationship will foster a deeper understanding of both the program's needs, and 
the potential solutions. [Refs. 39, 51] 

5. The PDSS Acquisition Management Process 

The reader may recall from Chapter II that software acquisition management is 
"...the process of acquiring custom software, usually by contract from some third party", 
and that it includes activities such as: planning, contracting, budgeting, evaluating and 
managing performance, and providing for the future support of the system. [Ref. 11, p. 
50] The sections below address the planning, budgeting, and the contracting aspects of 
the SO A PDSS acquisition. The current process of evaluating and managing performance 
will be addressed under Subsection 6 - The PDSS Project Management Process. As for 
providing for the future support of the system, this has yet to be addressed in any derail. 

a. Planning 

The primary planning tool utilized under the current PDSS program is the 
statement of work (SOW). It defines the scope of the software effort to be accomplished 
by LORAL for the particular year of the contract. [Refs. 52, 53] The SOW is developed 
by the PM in conjunction with his one matrix software engineer and input from the user 
pilots and SIMO. In both the base and first option year, the Government had to revise the 
SOW during the proposal/negotiation process. This was done so that the scope of work 
could be accomplished within budget. The SOW establishes two separate processes for 
PDSS during the year of support. The first process is the development of the new source 
software release for the IAP. The second is a process to handle the correction of priority 
one and two STRs which may occur during the year. These are software problems which 
either prevent the aircraft from performing its mission, or which severely degrade mission 
performance. 

The key tools used in the first process, defining the content of year's new IAP 
software release, are the databases which are maintained to track software trouble reports. 
The first database, the Government Trouble Report (GTR) database is maintained by 


58 




SIMO and documents trouble reports. These are problems with the software reported by 
the user pilots. The second database, the Common Database (CDB) is maintained by 
LORAL and documents all STRs, both GTRs and those found during LORAL's software 
development and testing effort. These databases provide a common framework with 
which to define what new functionality will be added to the current fielded baseline IAP 
software. For example, the SOW for the base year of the contract, [Ref. 52] which 
yielded IAP Source Release 12.0 specified those specific GTR numbers which were to be 
implemented into the new source release. 

The second process is the correction of any priority one or two STRs. This is 
accomplished as a separate effort, "over and above" the base effort conducted to produce 
the IAP source release. Under this process, LORAL begins investigation of the reported 
STR within 24 hours, and recommends corrective action to the PM within five calendar 
days. The recommendation includes an estimate of the cost and schedule to implement a 
correction, and an estimate of the impact the correction will have on the system. If 
approved, LORAL takes action and implements the correction. 

While these are the two main efforts included in the SOW, it also addresses the 
following topics: 

• Software documentation. 

• Support of flight testing of the new source release. 

• Program management reviews. 

b. Budgeting for PDSS 

The PDSS effort is funded primarily with Major Force Program (MFP) 11, 
Operations and Maintenance (O&M) funds. Funds are allocated to TAPO for PDSS 
through the U.S. Army Special Operations Command. Currently the SOA PDSS program 
is operating on funds which were budgeted several years ago by PM SOA. Future years 
are budgeted by SIMO in accordance with the Planning, Programming and Budgeting 
System (PPBS). Under this process, SIMO is currently working on the budget estimate 


59 



for the Program Objectives Memorandum (POM) years 1998 through 2003. In 
developing budget projections, SIMO considers two aspects of PDSS. First it considers 
basic software support, funded with O&M funds. It projects this amount based on what 
has been spent to date, plus an inflation factor. Second, it considers projected system 
enhancements, such as integration of new avionics components. For these efforts, it 
budgets research and development funds (RDT&E), procurement funds for purchase of 
new hardware components, and O&M funds for software integration. Estimates for these 
future efforts are essentially "best guesses." [Ref. 54] 

c. The PDSS Contract 

The current PDSS contract was awarded on 25 July 1994 as a 
firm-fixed-price contract, with two option periods. The period of performance for both 
the base and the options is one year. A firm-fixed-price contract was selected essentially 
to limit the Government's risk in the correction of any software errors discovered during 
TF development of the MMR. At that time it was not known how large an effort it would 
be. The fear of this risk was motivated at least in part from events which occurred during 
the software development, when there were disagreements among the ACs, IBM, and the 
Government over who had the responsibility to correct software anomalies which were 
considered "out of scope" under the firm-fixed-price development and production 
contracts. [Ref. 55] Under the first year of the contract, the over and above line items for 
correction of priority one and two STRs, and other software efforts were also 
firm-fixed-price. In the first option year, these line items have been changed to a time and 
materials (T&M) basis. An example of this is the effort to integrate a new Digital Map 
processor into the IAS. This effort, estimated to cost $2 million will be accomplished on a 
T&M basis. 
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6. The PDSS Project Management Process 

Project management is the process by which the software development is 
monitored and controlled. Within the context of the SOA PDSS case, the areas of 
management reviews and metrics will be discussed. 

a. Reviews 

Together, the PDSS contract SOW [Refs. 52, 53] and LORAL's Software 
Development Plan [Ref. 40] establish a program of management reviews. These reviews 
and their purpose are discussed below: 

• Program Reviews - Program Reviews are held on an as needed basis for the 
purpose of discussing resolution of problems, methodology, documentation, and 
milestones. 

• Technical Interchange Meetings (TIM) - TIMs are held periodically during the 
software development cycle to discuss the implementation, the possible rework, 
or the deletion of the GTRs which were specified to be included in the source 
software release. 

• Flight Readiness Review (FRR) - The Flight Readiness Review is conducted 
prior to flight testing of the software release to review the Flight Test Plan. 
Following flight testing of the software, a Flight Test Report is generated by 
LORAL. Acceptance of this report documents the Government's acceptance of 
the software. 

• User Lab Tests - In addition to the reviews formally required in the SOW, User 
Lab Tests have been conducted periodically during software development to 
obtain user input on the way in which GTRs are being implemented. [Ref. 56] 

b. Metrics 

The only metrics called for in the PDSS SOW is maintenance of the 
Common Database. LORAL does collect and periodically provides metrics concerning 
the growth in the size of the IAP, and processor and memory utilization, but there is no 
formal metrics program required in the SOW. 
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7. The PDSS Systems/Software Engineering Process and Tools 

This section describes the systems and software engineering processes, and the 
software engineering tools in use on the SOA PDSS program. Software engineering is the 
"...technological and managerial discipline concerned with systematic production and 
maintenance of software products..." [Ref. 3, p. 5-1] Systems engineering is defined as: 

...the management function which controls the total system 
development for the purposes of achieving an optimum balance of all 
system elements . It is a process which transforms an operational need into 
a description of system parameters and integrates those parameters to 
optimize the overall system effectiveness . [Ref. 57, p. 1-2] 

a. The Systems/Software Engineering Process 

Support of the SOA IAS is more than simply an exercise in computer 
programming. As a highly complex, integrated system consisting of both hardware and 
software, systems engineering is a critical function in the management of the system as it 
grows and evolves through the system life-cycle. Systems engineers are responsible for 
assessing the impact of any changes to be made in the software and /or IAS hardware, for 
making the required changes to the Software Requirements Specification (SRS), and for 
conducting the system level integration and testing. [Ref. 40, p. 103] Other systems 
engineering activities are shown in Figure 16. The software engineering process 
essentially mirrors the process described in DoD Standard 2167A. While LORAL did 
begin to incorporate some prototyping in the development of IAP Release 12.0, the 
process can be characterized predominantly as a waterfall process, similar to that shown in 
Figure 5. 

b. Software Engineering Tools 

The SOA IAS was developed and is maintained on multiple software 
toolsets. Recall from Figure 14, that of the nine CSCIs in the IAS, LORAL developed 
and maintains only the IAP and the SROMP at its facility; the other CSCIs are maintained 
by LORAL through subcontracts. The IAP, the heart of the system was developed in 
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JOVIAL. This is an older third generation Higher Order Language (HOL). The toolset 
on which it was developed is also old technology, dating from the 1980s and cannot 
support many of the recently developed capabilities in software engineering. LORAL's 
major IAS subcontractor is Allied-Signal Aerospace Company, Flight Systems Division. 
The Allied CSCIs include the DP, RTU, and the MDG. Figure 14 also shows that these 
CSCIs were developed in several languages, including JOVIAL, Ada, and assembly 
language. Assembly languages are second generation programming languages which 
differ for each type of computer processor. The CDU CSCI was developed by Smith 
Industries Aerospace in assembly language. Thus, the IAS development and maintenance 
is conducted on a distributed architecture. This involves activities at three contractors, 
with six separate languages and the tools to support them. [Ref. 7] 
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8 . 


PDSS Year One 


The base year of the PDSS contract was awarded as an undefinitized contract 
action on 25 July 1994. It was negotiated by the administrative contracting officer, and 
definitized on 25 April 1995. The primary effort under the contract was to develop and 
deliver IAP Release 12.0. The content was defined as a set of approximately 40 GTRs, 
most of which would be classified as user enhancements. As a risk limitation measure 
under the firm-fixed-price contract, LORAL required a limit of 1,675 source lines of code 
(SLOC) to be placed in the SOW. However, LORAL implemented the functionality 
required by the GTRs within the firm-fixed-price of $4.45 million, but far exceeded the 
SLOC estimate of 1,675, delivering approximately 5,400 SLOC. 

9. PDSS Year Two 

The SOW for the first option year of PDSS calls for the development and delivery 
of IAP Release 13.0. The major task for this source release will be to implement the 
software "patches" which were developed during the development and qualification of the 
TF capability of the MMR. Patches are essentially separate programs which run with the 
IAP, but are not compiled in the IAP source code. The software changes for Release 13.0 
have already been developed in these patches, but will have to be implemented in the IAP 
source code, and then integrated and tested. In addition to this effort, on a time and 
materials basis, a new LRU called the Digital Map Processor will be integrated into the 
IAS, replacing the current Map Display Generator. This effort is estimated to cost $2 
million and is outside the base effort. 

Negotiations for the first option year took place in late August and early 
September 1995. Because the effort was funded with expiring O&M funds, there was 
great pressure to negotiate the price for the option quickly. At the first round of 
negotiations, the PM's supporting software engineer, and the MICOM software engineers 
presented the Government's technical position on the effort. The technical position was 
based on the engineers' analysis of LORAL's proposal. The Government's technical 
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position on labor hours, and its cost position were approximately half the amount 
proposed by LORAL. From LORAL's perspective, the Government's position was so 
dramatically different from its own that it represented a different scope of work than that 
specified in the SOW. LORAL's program manager expressed the following concerns 
about the Government's position and the PDSS effort in general: [Ref. 58] 

• The engineering hours in the Government position would require LORAL to 
expedite delivery of Release 13.0 in December 1995. SOA software 
development personnel would then have to be moved to other projects, leaving 
the program without dedicated support until additional over and above efforts 
were funded. 

• The Government position calls for the elimination of software verification 
testing. This activity ensures that changes in the CSCIs are traced back to the 
software requirements specification. 

• The exit criteria for flight testing, which documents acceptance of the software 
is not clear. LORAL faces what it sees as an unbounded risk under the 
firm-fixed-price contract for investigation of all STRs found during flight testing 
of Release 13.0. In past flight tests, most of the STRs reported by the pilots 
have not been LORAL's responsibility. 

Following this initial meeting, LORAL negotiators returned to the plant to develop 
a revised proposal. Negotiations were settled approximately a week later at a 
firm-fixed-price of $2.8 million. This price provides what LORAL called a "framework" - 
which essentially keeps the SOA development staff committed to the program for the 
entire year of support. 

D. CHAPTER SUMMARY 

The Special Operations Aircraft program began in response to the tragic 
circumstances which caused the failure of the Iranian hostage rescue mission in April 
1980. As the White House Statement shown at the beginning of the chapter indicated, the 
mission was terminated "...because of equipment failure." Today, the MH-60K and 
MH-47E Special Operations Aircraft are equipped with a Mission Equipment Package 
(MEP) which allows the aircraft to operate reliably under the demanding conditions 
encountered by the aviators of the 160th Special Operations Aviation Regiment. The 
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heart of the MEP is the software intensive SOA Integrated Avionics Subsystem (IAS) 
which reduces pilot workload in mission planning, navigation, and communication. As the 
IAS continues to evolve in response to changing user requirements, the effective 
management of the software system through PDSS is essential to ensure that there is 
never a repeat of "Desert One." 
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IV. ANALYSIS AND LESSONS LEARNED 


A. INTRODUCTION 

This chapter provides the analysis of the SOA PDSS case described in Chapter III. 
The objective of this analysis is to identify the significant management issues present in the 
SOA PDSS case and to present the lessons learned from analysis of these issues. The 
reader will recall from Chapter II, that software acquisition management involves the 
interaction of the buyer, the seller, and the user in three overlapping processes as shown 
below in Figure 17. 



Figure 17. The Software Acquisition Management Relationships 

From [Ref. 11, p. 51 j 

The acquisition management process is primarily the responsibility of the PM and 
his supporting contracting officer. The software engineering process is primarily the 
responsibility of the software development contractor, and both the PM and the contractor 
share responsibility for the project management process. This model provides a 
convenient framework within which to identify and analyze the significant issues which 
impact the management of the SOA PDSS program. 
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B. ACQUISITION MANAGEMENT ISSUES 


The acquisition management process includes those activities undertaken by the 
PM to plan, budget, and contract for the software acquisition. The significant acquisition 
management issues identified in the SOA PDSS case are in the areas of cost, program 
plans, the buyer-seller relationship, and the contract type. These issues are analyzed in the 
following sections. 

1. Cost 

The most significant underlying issue involved in the SOA PDSS acquisition, from 
the Government's perspective, is the cost of the PDSS contract. The cost issue is 
characterized by a Government perspective that the LORAL PDSS contract costs too 
much and is inefficient. This perception is the cause of considerable friction between the 
Government and the contractor. This friction in turn has fostered a harmful short-term 
perspective, and mistrust on the part of the Army (both the PM and the user) in regard to 
the buyer-seller relationship. 

The cost of the PDSS contract is presently viewed as too high by the PM. This 
research however, leads one to ask: By what standard is the PDSS cost too high? The 
PDSS contract is essentially a new effort for LORAL. As such, there is no valid basis for 
comparison of the PDSS cost to LORAL's historical development cost data. The Defense 
Contract Audit Agency has confirmed this. In its audit of LORAL's base year PDSS 
proposal, it stated that because the software maintenance effort is essentially new, 
historical analysis of the proposed labor cost could not be accomplished. [Ref. 59] 

One question which could be asked: Is the cost too high in comparison with what 
other aviation programs pay for PDSS? Since the SOA is a unique system, it is difficult to 
compare it with other aviation systems. For example, the Navy's LAMPS helicopter 
program, also supported by LORAL, spends about two million dollars per year for PDSS. 
The SOA's base year PDSS cost was $4.45 million. [Ref. 60] One cannot simply 
compare costs. The software systems are different in purpose and scope. 
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A second question: Is the cost too high in relation to the Government's own 
estimate of the effort required to implement the requested changes? When one examines 
this issue it becomes clear that the Government does not currently have the system 
knowledge, expertise, or capability to perform its own detailed analysis of what changes to 
the IAS software should cost. The PM's matrix support software engineer, and the 
MICOM software engineers were effective in negotiating a lower price for the first option 
year of PDSS. However, this success was based upon cutting the scope of work, and on 
decrements to LORAL's proposal. It was not based on an independent cost estimate of 
the effort specified in the SOW. Estimates of this nature require both software 
engineering expertise, and detailed system specific expertise. While the PM's supporting 
software engineers are experts, the detailed system specific expertise currently resides only 
in LORAL. 

Another question: Is the cost too high in relation to what was budgeted for 
PDSS? The answer - perhaps. The budget for a given year's PDSS effort is prepared at 
least two years in advance of the actual budget year. At such a time, the extent of the 
changes to be made to the system two years in advance were not forecast in detail. As a 
result, it is not surprising that the budget does not support the purchase of all of the new 
functionality now desired. 

From a Government perspective, the primary complaint is with regard to the 
categories of cost which are viewed as "non-value added" as compared to the direct effort 
of software support. These include categories such as proposal preparation and program 
management, which accounted for 30 percent of the base year cost. The distribution of 
the actual base year PDSS costs, and the proposed option year 1 costs are shown in 
Figure 18. While generally dissatisfied with the proportions of these cost categories, the 
Army must admit that at least part of the responsibility for such costs lies with the 
Government. 
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Figure 18. PDSS Cost Distribution 
From [Ref. 61] 

For example, proposal preparation costs, at eight to 15 percent of costs, are 
partially impacted by the several iterations of SOWs to which the contractor has typically 
had to react before the Government finally defines the scope of the PDSS effort. While 
program management costs do appear to be high, there has been no direct effort on the 
part of the Government, other than during negotiations to reduce it. From LORAL's 
perspective, this category of cost is required and simply reflects the level of management 
necessary to run the SOA PDSS program. This may be an area which could be addressed 
in an incentive type contract. A closer, more cooperative working relationship with 
LORAL could also significantly impact this area. 

In addition to these areas, which the Government views as non-value added, the 
cost of PDSS is also necessarily a function of the scope of the effort specified in the SOW. 
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Three of the major tasks specified in the SOW are the IAP software release, software 
documentation, and the DP source release, a subcontractor effort. 

Early in the development cycle, during the PDSS base year, the SOA PM 
Technical Division Director performed an informal analysis of the content of IAP release 
12.0. This analysis revealed that approximately 87% of the additional function added to 
the software was in the form of user requested enhancements. [Ref. 62] The remainder 
was in the form of corrective actions. Such system improvements add real value to the 
system. 

A major factor that impacts the cost of PDSS is the amount of documentation 
which the Government buys. Documentation is defined as the "paperware" which, 
"...provides the means to describe the product and the engineering which lead to its 
development." [Ref. 19, p. 24] Figure 18 shows that documentation accounted for ten 
percent of the actual cost of the base year contract. For the first option year, 17 percent 
of the contract price will go toward documentation. The cost of documentation is 
impacted by the fact that the future direction for PDSS after the current contract is still in 
question (i.e., if the Government plans to take over PDSS, or transition PDSS to another 
contractor, detailed documentation will be required). If a PDSS strategy is confirmed, 
then the type and amount of documentation could be tailored specifically to support the 
selected strategy. The third area of cost directly resulting from the scope of work 
specified in the SOW is the DP source release. This cost is shown in Figure 18 as 
subcontract cost. 

Thus, the total cost of PDSS is a function of not only the software products 
developed, but also of the joint Govemment-LORAL process for arriving at that product. 
This view is confirmed by researchers Haimes and Chittister. [Ref. 63] In research 
concerning the management of cost and schedule risk in software development, they 
concluded that the sources of risk in these areas can be attributed to both the buyer and 
the seller. 


71 



In terms of the contractor, several of the factors which contribute to cost and 
schedule risk are: [Ref. 63, pp. 133-134] 

• Organizational maturity level. 

• The process and procedures followed in the assessment of the project's cost and 
schedule. 

• The level of honesty exhibited by management in communicating the real cost 
and schedule to the customer (and of course vice versa). 

• The level of experience and expertise of the staff engineers in software 
engineering, in general, and in the application domain in particular. 

• The level of experience and expertise of the management team in software 
engineering. 

• Financial and competitive considerations. 

In terms of the buyer, Haimes and Chittister found that the following factors 
influence cost and schedule risk: [Ref. 63, p. 134] 

• The process and procedures followed in the assessment of the project's cost and 
schedule. 

• The level of specificity at which the system and software requirements are 
detailed. 

• The number of changes and modifications requested by the customer during the 
software development process which are often not harmonious with earlier 
specification requirements. 

• The commitment of the customer's project management to monitor and oversee 
the software development process. 

• The level of honesty exhibited by management in communicating the real cost to 
the "real client," the U.S. Congress. 

The true issue in regard to cost is not its relative magnitude. Rather, it is how it is 
jointly managed by the Government and LORAL. Both parties have an interest in 
maintaining an affordable PDSS program. The Government's interest is the continuity of 
affordable, quality software maintenance support. LORAL's interest is in satisfying the 
SOA customers and keeping the SOA business. The issue of cost is critical because it 
influences, or is influenced by nearly all of the other issues in the SOA PDSS program. 
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2. Program Plans 

Development of program strategies and plans is one of a PM's major acquisition 
management functions. Program plans establish the overall framework for the 
management of the program. Currently, the only acquisition planning mechanism in use in 
the SOA PDSS acquisition is the SOW for the current year's PDSS effort. This is a 
significant issue because lack of a longer range plan fosters a harmful short term 
management perspective. The key program plan in effective management of PDSS is the 
Computer Resources Life-Cycle Management Plan (CRLCMP). [Ref. 13, p. 6-D-2] The 
CRLCMP serves to document the PDSS concept or strategy, and the detailed approach to 
implement that concept. The existing SOA version of the CRLCMP, the CRMP was last 
updated in June 1992. It reflects the former CECOM support concept discussed in 
Chapter III. This concept called for the purchase of a complete SEE with which to 
maintain all of the IAS CSCIs. Transition from LORAL to CECOM was to have been 
accomplished over several years, during which LORAL's participation would be gradually 
phased out. 

The PM fully recognizes the need for a long range strategy for PDSS. However, 
as essentially a one man operation, he has little time to spend, and few support resources 
to assist in the development of an acquisition strategy. The importance of acquisition 
strategy and planning is firmly supported in policy. The term "acquisition strategy" refers 
primarily to the formal strategy required by the DoDI 5000.2 for a major acquisition 
program. However, a related concept which is more appropriate in this case is that of 
acquisition planning. 

In reference to acquisition planning, FAR Part 7.101 states that: 

"Acquisition planning" means the process by which the efforts of all 
personnel responsible for an acquisition are coordinated and integrated 
through a comprehensive plan for fulfilling the agency need in a timely 
manner and at a reasonable cost. It includes developing the overall 
strategy for managing the acquisition. [Ref. 64, Part 7.101] 


73 


This issue seriously impacts the SOA PDSS program because failure to identify the 
PDSS program's future direction impacts the decisions presently being made. An example 
of this is the decision of how much, and what type of documentation to buy. Currently, 
the PM and the user appear to be contemplating two different directions for the future of 
the PDSS program after this current contract is completed. The user, although satisfied 
with the quality and the performance of the software, is very dissatisfied with its cost. As 
a result, the user wants to procure PDSS services competitively. In contrast, it appears 
that the PM recognizes that transition to another source would be a risky venture. The 
PM would prefer to continue working with LORAL, at least until the Government reduces 
its risk through the development of a base of expertise in the SOA IAS software, and in 
general learns to manage software acquisition. 

A competitive procurement conducted solely in the "hope" of finding a cheaper 
source would be extremely risky, and may be "penny wise and pound foolish." There are 
two sources of risk in this area. First is the risk in maintaining working relationships with 
LORAL's IAS subcontractors, as well as the relationship with LORAL on the SOA Life 
Cycle Contractor Support (LCCS) Contract. The reader will recall from Chapter in that 
seven of the nine IAS CSCIs were developed and are currently maintained by 
subcontractors. The Government has depended on LORAL to manage these 
subcontractors in maintaining the IAS. How would changing sources impact these 
relationships and the program's ability to maintain responsive support from these 
subcontractors? Additionally, LORAL is a major subcontractor to BSAS. As such it is 
responsible for the depot level maintenance of the AP102A MP hardware. How would 
changing sources impact operations on that contract? 

The second source of risk in changing PDSS sources concerns the complexity of 
the IAS. Because of the complexity of the SOA IAS, and the strategic importance of the 
user's mission, systems and software engineering expertise is critical in maintaining the 
system. This point has been illustrated in at least two other complex, software intensive 
DoD systems. The Navy's LAMPS program transitioned software support from LORAL 
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to a Navy SSA. It has since elected to transition back to LORAL. The primary reason for 
this reversal was for the systems engineering expertise at LORAL. A second reason was 
that the cost of maintaining two software engineering environments (i.e., one at LORAL 
and one at the SSA) was prohibitive. These costs included: maintenance of dual sets of 
avionics components and laboratory hardware; and the development tools. [Ref. 60] The 
Army's PATRIOT Missile system underwent a similar transition. It also has since 
transitioned support back to the original software developer. 

The PATRIOT'S software engineering chief made this comment concerning the 
folly of pursuing the lowest bidder for software development: 

It is false economy in software development to go get the low 
bidder (Government or contractor) to do your software. In complex 
systems, if the low bidder doesn't have the requisite expertise or experience 
he will encounter major problems and costs will soar. In many areas, ten 
people who lack system specific experience simply cannot do the critical 
job that one experienced person can do. Throwing cheap people at 
complex software jobs is not a viable answer. [Ref. 65] 

The SOA PDSS process action team (PAT), charged with the original PDSS 
assessment included user representation. It also recognized the importance of system 
specific expertise. In considering which PDSS alternative to recommend to the PM SOA, 
it considered the following factors: [Ref. 8] 

• Maintenance Experience - The amount of maintenance experience in the 
organization. The maintenance task differs from that of development, and prior 
experience in maintenance was considered a benefit. 

• SOA Experience - SOA experience consists of "domain" knowledge of the SOA 
software. Prior or current SOA software experience was considered a benefit. 

• No Transition - Transition is a description identifying the shift from the original 
software developer/maintainer. A loss of knowledge and familiarity as well as 
domain expertise is to be expected during this shift. Because of this, not 
transitioning was considered a benefit. 

• Aircraft Knowledge - This attribute captures the amount of knowledge of the 
total aircraft system. This includes experience with the interaction and 
dependencies involved with the airframe and components, propulsion, pilot, and 
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flight envelopes. Configuration management of the aircraft system was also 
considered in this attribute. 

If these factors were important in 1993, are they any less important now? Clearly, 
these factors are as important now as they were when the PAT considered the various 
PDSS options in 1993. It was these factors that lead to selection of LORAL to perform 
PDSS, despite the higher cost of that alternative. 

While the PM lacks a written acquisition strategy or plan, this is but one dimension 
of the concept of strategy. Another dimension of strategy addressed by author Henry 
Mintzberg [Ref. 66, p. 13] is that strategy is also a pattern. In this sense, strategy is a 
pattern in a stream of actions. Under this definition, strategy is not a plan, but rather is 
represented by consistency in behavior, whether or not intended. In the researchers view, 
there is a definite pattern of actions which indicate a PDSS strategy which centers on 
LORAL. This pattern indicates growing dependence on LORAL as the SOA systems 
integrator, and thus as the only firm (at this time) that can realistically accomplish the 
enhancements which the user requires during PDSS. The pattern of actions includes: 

• LORAL's role as a key LCCS subcontractor. 

• The selection of LORAL to accomplish the integration and qualification of the 
TF capability of the mulitmode radar. 

• Departing from the original PDSS concept which involved a transition to 
Government support at CECOM, to a contract with LORAL. 

• A continued stream of system enhancements implemented by LORAL. These 
include the integration of a new digital map processor, and the embedded GPS 
inertial. These enhancements further complicate the original system. 

• Avoiding the risk reduction actions to prepare for transition to another source. 
Such actions might include a transition assessment, the use of an independent 
verification and validation contractor, and development of a transition plan. 

While there is no doubt that another source could eventually take over the PDSS 
effort, one must ask, what would be the price (in terms of cost, quality, and 
responsiveness of support)? This is not to say that SOA must always be supported by 
LORAL, but rather that transition is risky and such risk must be confronted. There are 
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unknown costs and risks involved in changing sources, and a hasty transition could 
potentially leave the SOA IAS without support. A transition to another source would 
require detailed planning, and a coordinated sequence of hand-offs as specified in the 
discussion of Pre-Deployment Software Support in Chapter II. 

The second issue in regard to program plans is the planning of the evolution of the 
IAS. This includes establishing a link between the desired system changes and 
enhancements, and the budget. The program requirements, in the form of enhancements, 
(such as the Digital Map Processor integration) for the base and first option year had not 
been tied in any recognizable way to the PDSS budget. Since detailed Budget Estimate 
Submissions are sent into the Planning, Programming, and Budgeting System (PPBS) two 
years prior to a given budget year, [Ref. 67] PDSS requirements must be identified, 
estimated, and budgeted at least that far in advance. According to a SIMO representative, 
the PDSS budget is now being planned in accordance with the PPBS. SIMO is currently 
working on the POM for the years 1998 through 2003. However, without SOA 
knowledgeable system and software engineers working on these budget estimates, their 
accuracy is in question. Additionally, there is no single integrated plan which documents 
planned IAS enhancements and the corresponding budget. This is an issue because the PM 
cannot make long range plans without visibility into the future requirements and the 
proper budget to implement those requirements. 

Another dimension of this issue is the type of funds used for PDSS. It is 
understood that maintenance of the current software baseline is accomplished with O&M 
funds. However, the addition of new functions or hardware which create a new baseline 
should be implemented with the proper combination of RDT&E and procurement funds. 
The integration of the new digital map processor can serve as an example. Both the 
purchase of any new avionics components and their integration/software development 
should be accomplished with RDT&E and/or procurement funds. Use of O&M funds for 
integration of new hardware, distorts the true cost of supporting the existing software 
baseline. 
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In summary, there are two major issues in regard to planning on the SOA PDSS 
program. First is the lack of a clear acquisition strategy for PDSS after the current 
contract. This lack of clear direction fosters a harmful short term management 
perspective. The second major issue is the planning of the evolution of the IAS. This 
includes the budget required to implement system enhancements. This is an issue because 
the PM cannot make long range plans without visibility into the future requirements and 
the proper budget to implement those requirements. 

3. The Buyer-Seller Relationship 

One of the most serious issues in regard to the SOA PDSS acquisition is the level 
of trust and cooperation in the buyer-seller relationship. While the Government and 
LORAL are dependent on each other in this relationship, the present level of trust and 
cooperation does not reflect this level of mutual dependence. As a result, issues such as 
cost, (which could potentially be addressed through a closer working relationship), are left 
unsolved. One indication of the level of mistrust is the selection of a firm-fixed-price 
contract. It places all of the cost risk on LORAL for effort that is largely developmental. 
Much of this mistrust on the part of the Government, specifically among the user 
community, is left over from the SOA development and production programs. During 
those programs, there were disputes over exactly what functionality the PIDS required, 
and which of the contractor's, (the ACs or LORAL) would be responsible for making the 
required changes to meet the user's desires. All of this was the direct result of the fact that 
all contracts in the program were firm-fixed-price, and did not reflect the obvious level of 
development risk to all the parties involved. 

The same situation is present to a degree in the PDSS contract. The situation 
under the current firm-fixed-price contract finds the Government attempting to get all the 
additional software functionality it can within its budget. And LORAL attempting to 
protect itself from the risks involved in proposing firm-fixed-prices for development of 
new software functions. This situation is almost certain to create an adversarial 
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relationship. This puts the parties further apart than the required "arms length 
relationship." 

The benefit of closer working relationships between the Government and its 
contractors is supported both in theory and in policy. In examining the theoretical 
approaches to the buyer-seller relationship in Defense contracting, Lieutenant Colonel 
Carl R. Templin [Ref 68] examined the degree of "coupling" between the buyer and the 
seller. Based on the research of Landeros and Monczka [Ref. 69], the theory examines 
three degrees of coupling in the buyer-seller relationship. The degree of coupling present 
in the relationship is defined by attributes such as: 

• The number of suppliers available to the seller; 

• The degree of commitment or "alliance" between the parties; 

• How the parties resolve disputes; 

• The flow of information between the parties, and; 

• How the parties adjust to a changing market. 

These factors and the degree of coupling in the relationship are depicted 
graphically in Figure 19. The model in Figure 19 shows the degree of coupling between 
the parties may vary on a continuum, from loosely coupled to fully coupled. On the left of 
the continuum, a loosely coupled relationship is one in which the independence of the 
parties is maintained through open market bargaining. In DoD this is characterized by a 
contractual relationship based on sealed bidding, where price is the sole source selection 
factor, and the contract type is firm-fixed-price. On the right side of the continuum is a 
fully coupled, or vertically integrated relationship. A DoD example of this is an organic 
Government software support activity. 

Between these two extremes are tightly coupled relationships. These relationships 
are based on cooperative buyer-seller relationships which are designed to achieve mutually 
beneficial long term strategic goals. These goals may include cost reduction, better 
performance, and improved quality and reliability. [Ref. 68, p. 124] This type of 
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relationship is characterized by a considerable degree of interdependency between the 
parties. Because of this degree of interdependency, there is also a high degree of 
cooperation. Comparing this theoretical model to the relationship between the 
Government and LORAL, it appears that there is a mismatching of the degree of coupling 
and the attributes which define the situation. 


Attribute: 

Tightly Coupled 

Loosely Coupled (Cooperative 

(Market Bargaining) Relationship) 

Fully Coupled 
(Vertical Integration) 

Supply Pool 

Numerous 

Suppliers 

- -- 

One 

Supplier 

Alliance 

Credible 

Threat 

- -- 

Credible 

Commitment 

Dispute 

Resolution 

Unyielding 

Negotiations 

- -► 

Managerial 

Tradeoffs 

Information 

Exchange 

Minimal 

--- 

Great 

Marketplace 

Adjustment 

Separate 

- - - 

Joint 


Figure 19. Buyer-Seller System Coupling 
From [Ref. 68] 


In this situation, the degree of interdependence between the Government and 
LORAL is high. LORAL is dependent on the SOA business to keep personnel employed. 
The Government is dependent on LORAL's SOA system knowledge and expertise. This 
interdependence indicates that a tightly coupled relationship, in the center of the 
continuum is appropriate in this situation. However, several of the attributes of the 
relationship indicate that the parties believe that this a loosely coupled relationship, able to 
be controlled by market bargaining. 

This is demonstrated by several of the attributes which define the relationship. 
First the Government believes that there are numerous suppliers available which could 
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compete for the PDSS contract and therefore drive down the cost. In fact, at this time in 
the program, the pool of realistically qualified suppliers available to the Government is 
one - LORAL. Secondly, the level of commitment or alliance is low. The Government's 
use of the "going competitive" threat indicates a low level of commitment in the 
relationship. Thirdly, the level of information exchange, especially in the area of software 
process metrics is low. Application of this model indicates that the relationship would be 
most effective with increased cooperation. But instead, it is presently managed as a 
loosely coupled relationship. 

The value of cooperative buyer-seller relationships is also supported in policy. The 
recently approved Statement of Guiding Principles for the Federal Acquisition System 
[Ref. 70] seeks an environment which fosters cooperative relationships between the 
Government and its contractors, consistent with the overriding responsibility to the 
taxpayer. 

Thus, the level of mistrust and the lack of cooperation in the buyer-seller 
relationship is a major issue. It prevents joint problem solving of issues which impact both 
sides of the relationship. 

4. Contract Type 

The single most critical document in the acquisition of weapon systems software, 
or in this case the acquisition of software support, is the contract itself. The contract 
defines the legal relationship and the obligations which exists between the buyer and seller. 
Selection of the contract type is generally a matter for negotiation and requires the 
exercise of sound judgment. The negotiation of the contract type is closely related to 
negotiation of the price and the two elements should be considered together. The 
objective is to negotiate a contract type and price which reasonably allocates risk between 
the contractor and the Government, and provides the contractor with the greatest 
incentive for efficient and economical performance. [Ref. 64, 16.103(a)] 
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In selecting a contract type, a contracting officer should consider these factors: 
[Ref. 64, 16.104]: 

• The degree of price competition present. 

• The degree to which price analysis can provide a realistic pricing standard. 

• In the absence of adequate price competition or if price analysis is not sufficient, 
the degree to which analysis of the contractor's estimated costs will provide a 
sound basis for negotiating prices. 

• Type and complexity of the requirement. 

• Urgency of the requirement. 

• Period of contract performance. 

• Contractor's technical capability and financial responsibility. 

• Adequacy of the contractor's accounting system to support contract types other 
than firm-fixed-price. 

• The impact of any concurrent contracts. 

• The contractor risks due to the amount of subcontracting involved. 

Considering these factors, the use of a firm-fixed-price (FFP) contract for the SOA 
PDSS presents a significant acquisition management issue for several reasons. First, it 
does not equitably distribute the cost risk involved in this complex effort. While this 
contract is for software "maintenance," much of the effort accomplished thus far has been 
enhancements, or modifications. These enhancements amount to new development, and 
therefore do involve considerable risk in estimating the amount of effort required to 
implement them. This risk was evident in the use of the 1675 SLOC limit in the PDSS 
base year. LORAL's insistence on the limit, and then vastly exceeding it indicates that 
even with its SOA experience, it is difficult for the contractor to accurately estimate effort 
for changes. 

Another area of cost risk, discussed during the negotiations for the first option 
year of PDSS was in the flight testing of the annual software release. [Ref. 58] Cost risk in 
this area results from what LORAL believes are unclear test exit criteria, and Government 
requests to make corrections beyond the scope of the new changes that were implemented 
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in the software. During the negotiations for the first option year, [Ref. 58] LORAL 
negotiators stated that in the past 96 percent of flight test STRs were in this category. 

Second, as a sole source contract, the competitive forces of the market place are 
not in effect controlling prices. Contributing to this is that the Government lacks any 
significant price history on which to establish price reasonableness. As stated previously, 
because SOA is a unique system, cost comparison with other systems is difficult. 

Third, software acquisition management requires increased buyer insight and 
visibility into the software development process. The use of a firm-fixed-price contract 
with its minimal administration does not support this increased visibility. 

Fourth, a firm-fixed-price contract does not support the PM's goal to control the 
cost of PDSS. Under the firm-fixed-price contract, the Government has the ability to 
influence cost only once during contract performance, and that is at the negotiation table. 

Finally, there appears to be confusion between the parties over exactly what the 
FFP portion of the contract buys. The Government sees the deliverable of the FFP 
portion of the contract being a software release. LORAL however, sees the FFP portion 
of the contract as a "framework" - what the researcher believes is a level of effort. This 
confusion may present problems later, especially in separating the costs of efforts 
accomplished under the FFP base effort, and any over and above efforts. 

In the PDSS contract's base year, both the base contract line item for the LAP 
source release, and the over and above line items, were priced on a firm-fixed-price basis. 
In the first option year, however, the over and above line items were changed to a time 
and materials basis. The time and materials pricing arrangement provides for the 
acquisition of supplies or services at a fixed hourly direct labor rate which includes wages, 
overhead, general and administrative expenses, and profit. Materials are priced at cost 
plus material handling fee if appropriate. The incorporation of a ceiling price does limit the 
Government's risk. [Ref. 64,16.601(a)] The integration of the new digital map processor 
will be accomplished on this over and above line. While this type of pricing arrangement 


83 




does take into account the risk in estimating the effort required to accomplish the 
integration, it provides no incentive for the contractor to control cost. This is not the 
most appropriate pricing arrangement for this situation. 

If these pricing arrangements are inappropriate, what are the options? The Federal 
Acquisition Regulation (FAR) addresses generally two broad categories of contract types, 
fixed-price contracts and cost-reimbursement contracts. These range from the presently 
used firm-fixed-price contract in which the contractor bears all of the cost risk to perform 
within the negotiated price, to the cost-plus-fixed-fee contract in which the Government 
bears the cost risk and the contractor's fee is fixed. 

Fixed-price contracts are those in which the price is fixed, or in some cases may 
adjust based on certain circumstances. In a fixed price contract, the contractor is bound to 
deliver the specified goods or services for the negotiated price, regardless of his cost 
performance. The Federal Acquisition Regulation (FAR) Part 16.2 lists the following 
fixed price type contracts: 

• Firm-Fixed-Price (FFP) 

• Fixed-Price with Economic Price Adjustment (FPE) 

• Fixed-Price Incentive, Firm Target (FPIF) 

• Fixed-Price Incentive, Successive Targets (FPIS) 

• Fixed-Price with Prospective Price Redetermination (FPRP) 

• Fixed-Ceiling-Price with Retroactive Price Redetermination (FPRR) 

• Firm-Fixed-Price, Level of Effort Term (FFP, LOE) 

In his study of contracting for software within the Department of the Navy, Naval 
Postgraduate School student Henry Attanasio found that in software acquisition, typically 
four types of fixed-price contracts are used. [Ref. 71, p. 29] 

• Firm-Fixed-Price - Under the FFP contract, the contractor is paid the 
negotiated price regardless of his cost experience in performing the work. It 
places on the contractor maximum risk and full responsibility for the resulting 
profit or loss. As such it also provides him with the maximum incentive to 
control costs and to perform in a manner which will yield the maximum profit. 
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This contract type requires minimal administration and is suitable for use in 
situations where the Government is acquiring commercial products or those 
which can be easily priced by competition or by comparison with similar 
products. [Ref. 64, 16.202] 

• Fixed-Price with Economic Price Adjustment - The FPE contract allows for 
an upward or downward revision of the contract price based on either: changes 
in established or published prices of certain items; adjustments in the actual cost 
of labor or materials; and adjustments based on certain labor or material indexes 
specified in the contract. This contract type may be useful when there is serious 
doubt concerning the stability of labor or material prices during the period of 
performance. The adjustment contingencies should be limited to those which are 
beyond the contractor's control. [Ref. 64,16.203] 

• Fixed-Price Incentive - The fixed-price incentive contract provides for 
adjusting profit and establishing the final contract price by application of a 
formula, or share ratio, which relates the final negotiated cost to a target cost. 
The final price is subject to a price ceiling which is negotiated at the start of the 
effort, and which serves to limit the Government’s risk. There are two forms of 
the fixed-price incentive contract: the fixed-price incentive firm target (FPIF), 
and the fixed-price incentive successive targets (FPIS). In both forms, the 
parties negotiate a target cost, a target profit, a price ceiling, and a profit 
adjustment formula at the outset. In the FPIS contract the share ratio is applied 
at different times during contract performance, while in the FPIF form, the share 
ratio is applied at the end of the effort. Since the contractor's profit is tied to the 
share ratio, he has the incentive to control cost and thereby raise his profit. The 
FPIF contract is appropriate when the parties can negotiate at the outset a firm 
target cost, target profit, and a profit adjustment formula which will provide for 
an adequate incentive to the contractor. The FPIS contract is appropriate when 
available cost or pricing data is insufficient to support negotiation of firm target 
cost and profit before award, but that data will become available early in 
production. [Ref. 64, 16.403] 

• Firm-Fixed-Price, Level of Effort Term - This contract type requires the 
contractor to expend a specified level of effort, probably stated in terms of labor 
hours, over a specified period of time, on work which is loosely defined, such as 
research. In return, the Government shall pay a firm-fixed-price for the 
contractor's effort. This contract type is suitable for research and usually results 
in the development of a report. However, payment is based on the effort 
expended rather than results of the effort. [Ref. 64,16.207] 

Cost-reimbursement contracts are for use only when the uncertainties in contract 
performance preclude estimating costs accurately enough to use one of the fixed-price 
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contract types. Under cost-reimbursement contracts, the contractor is paid all allowable 
costs under the terms of the contract, up to a specified ceiling which he may not exceed 
(except at his own risk) without the approval of the contracting officer. Since the 
Government agrees to pay all allowable costs, it bears most of the cost risk. The 
following are the common cost-reimbursement contracts. 

• Cost-Plus-Incentive-Fee (CPIF) - This cost-reimbursement contract provides 
for an initially negotiated fee which is then adjusted later based on the 
relationship between total allowable costs and target cost. The parties negotiate 
a target cost, a target fee, minimum and maximum fees, and a fee adjustment 
formula, or share ratio. After contract performance, the total fee is determined 
by application of the share ratio. The CPIF contract is appropriate for use in 
development or test programs in which a cost-reimbursement contract is 
appropriate and the parties can negotiate a target cost and a fee adjustment 
formula which will effectively motivate the contractor. [Ref. 64, 16.404-1] 

• Cost-Plus-Fixed-Fee (CPFF) - This cost-reimbursement contract provides for 
the payment of a negotiated fee which is fixed at the inception of the effort. The 
fee is negotiated based on a percentage of the estimated costs and does not vary 
in relation to actual costs, however, it may be adjusted if the scope of work to be 
performed is expanded. As the contractor receives a fixed fee regardless of cost 
performance, this contract type provides minimal incentive for the contractor to 
control costs. It is appropriate for use in research or exploratory studies where 
the level of effort required is unknown. [Ref. 64, 16.306] 

• Cost-Plus-Award-Fee (CPAF) - A CPAF contract is a cost-reimbursement 
contract which provides for the payment of a two part fee. The first part, called 
the base fee is fixed at the beginning of the effort and does not vary. The second 
part, called the award fee is an amount which the contractor may earn in whole 
or in part based upon the Government's judgmental, "after the fact" evaluation of 
the contractor's performance in terms of the criteria stated in the contract. 
These criteria may include areas such as cost control, quality, timeliness, 
management. The fee determination is made unilaterally by the Government and 
cannot be disputed by the contractor. [Refs. 64, 16.404-2 and 72,216.404-2] 

According to NASA, the agency which pioneered use of the CPAF contract, it is 
"...appropriate to use when cost uncertainty exists and key elements of performance 
cannot be objectively measured." [Ref. 73, p. 3] The contract is structured with 
evaluation criteria designed to motivate above minimum standards of performance 
established at the outset, and evaluation of performance by the Government at certain 
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intervals. This evaluation process often includes feedback from the contractor. 
Additionally, the Government has the ability to shift emphasis among the evaluation 
criteria during performance of the effort. A 1992 Air Force study [Ref. 74] concerning 
the use of award fees in software acquisition concluded that the award fee is, "...the most 
flexible provision in the FAR and its supplements to influence contractor performance 
during the (software) development process." [Ref. 74, p. 80] Another important benefit of 
this contract type is the increased communication between the Government and contractor 
as a consequence of the award fee evaluation process. These benefits were confirmed by 
the researcher in interviews with personnel responsible for the administration of three 
CPAF contracts used for software development or maintenance. [Refs. 75 and 76] 

A variation of the CPAF contract, is use of the award fee provision in conjunction 
with a fixed-price contract. This allows the Government to limit its risk with a negotiated 
fixed ceiling price, and also to motivate the contractor with the subjective evaluation of his 
performance. This variation was successfully used in a case involving the correction of 
software defects in the B-1B bomber. [Ref. 77, p. 26] 

C. PROJECT MANAGEMENT ISSUES 

Project management includes those activities which relate to the Government's 
management of the software development/maintenance process. The significant issues in 
this area are the lack of adequate project management resources, and use of software 
metrics. 


1. Project Management Resources 

Project management resources include systems and software engineering 
personnel, and the tools necessary to estimate software engineering effort. In the 
management of the PDSS program, the Program Manager essentially operates as a one 
man operation with whatever support he can muster through his personal initiative. This is 
an issue because he lacks the necessary technical support to adequately monitor the 
software development effort. This is especially critical in the area of analyzing the 


87 


potential impact changes will have on the IAS. Without knowledgeable, organic systems 
engineering support, the Government is totally dependent on LORAL for analysis of the 
impact system changes will have on the IAS. 

The second dimension of this issue is that currently, the PM does not have organic 
software engineering support. The lack of dedicated software engineering personnel in 
TAPO impacts project management in two respects. First, it forces the PM to personally 
manage the software development program. While the PM is an experienced SOA test 
pilot, he is not trained in software engineering. Additionally, the PDSS is but one of 
several programs which compete for his time. Second, without dedicated software 
engineering personnel, equipped with software cost estimation tools, the PM is unable to 
generate his own estimates for software development effort. This puts the Government at 
a great disadvantage when negotiating the firm-fixed-prices for the PDSS effort, or any 
over and above efforts. The TAPO organization has a billet for a software engineer, but a 
decision has been made not to fill this position. 

2. Software Metrics 

When you can measure what you are speaking about and express it 
in numbers, you know something about it; but when you cannot express it 
in numbers, your knowledge is of a meagre and unsatisfactory kind: it may 
be the beginning of knowledge, but you have scarcely, in your thoughts, 
advanced to the stage of science. 

This statement made by British scientist Lord Kelvin is appropriately displayed on 
the cover of the Navy's avionics metrics handbook [Ref. 78] The statement is certainly as 
timely now when discussing software project management, as when it was made over 100 
years ago. In contracting for PDSS with LORAL, the PM is buying a "process" as much, 
if not more than he is buying a "product." Measurement of that process, and its product 
is one of the first steps toward the PM's goal of becoming a "smart buyer." Metrics 
provide the "hard" evidence [Ref. 19, p. 71] or indicators of performance. They serve to 
highlight problem areas that require management attention. 
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The lack of a metrics program impacts both the Government and LORAL. It is a 
significant issue for Government for several reasons. First, it sends the signal to LORAL 
that the Government's only concern in the PDSS contract is the end product, the delivered 
software. Second, it denies the PM from having any objective basis with which to 
evaluate changes or improvements in the software development process, and in the quality 
of the delivered software. Third, without metrics data, any complaints that the user or the 
PM may have concerning LORAL's software processes are largely just unsubstantiated 
opinions. 

Lack of a metrics program is also a significant issue for LORAL. Without metrics, 
LORAL cannot objectively substantiate improvements it has made in the software process. 
For example, in the development of IAP release 12.0 (PDSS base year), LORAL began to 
incorporate aspects of the prototyping development methodology into the software 
process. While LORAL may view this as an "improvement" over a pure waterfall 
methodology, there is no way to measure the benefit without metrics data. This leads the 
researcher to ask the following questions: In what ways has this change improved the 
process? Have defects been reduced? Have specification changes been reduced? Has this 
change helped to lower the cost to the Government? Should the Government seek to 
expand the use of prototyping and if so why? 

D. SOFTWARE ENGINEERING ISSUES 

Software engineering is the technical and managerial process by which the 
software developer produces the products to be delivered under the contract. It consists 
of specific methods, procedures, and tools which assist in that process. There are three 
software engineering issues impacting the SOA PDSS program. They are the software 
development toolset and language, requirements definition and the software development 
methodology, and software process maturity and process improvement. 
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1. Software Development Toolset and Language 

The SOA IAP software was developed in JOVIAL, which is one of the older 
Higher Order Languages (HOL) used for computer programming. The toolset on which it 
was developed and is now maintained is also old technology. As a result of the age and 
limited capabilities of the toolset, functions which could be automated, are still done 
manually. Software configuration management is an example. Modem Computer Aided 
Software Engineering (CASE) environments for the DoD standard programming 
language, Ada allow for automating functions such as configuration management, 
documentation generation, and testing. Automation of these functions can significantly 
reduce the engineering labor hours involved in the software development process. 

2. Requirements Definition and the Software Development 
Methodology 

The waterfall development methodology in use at LORAL represents a significant 
issue when linked with a firm-fixed-price contract. Because it is difficult for the 
contractor to adequately define the detailed requirements for the given software product 
up front, there is significant cost risk under a firm-fixed-price contract. The fact that 
establishing the detailed requirements is difficult is recognized both by the software 
industry and by DoD. 

In his address to the annual DoD Software Technology Conference, Lieutenant 
General Otto J. Guenther, the Army's software executive made these remarks concerning 
software requirements: 

Where do I see opportunities for software improvement? ...allow 
requirements to evolve through the acquisition process. I don't think we 
can get the requirements 100 percent right initially in the acquisition 
process. What we need are adaptable requirements and an acquisition 
process for software intensive systems. [Ref. 79, p. 5] 

One way to incorporate adaptable requirements into the software acquisition 
process is through prototyping. Prototyping is essentially the process of building a 
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working replica of the software system. It may be used in conjunction with the waterfall 
methodology to demonstrate technical feasibility. In this sense it is also helpful in 
understanding and extracting the exact user requirements. The goal is to limit cost by 
understanding the problem before committing additional resources. [Ref. 80, p. 15] 

LORAL made use of the prototyping methodology to evolve requirements in the 
base year of PDSS. In the first option year, they also plan to use it to define the detailed 
requirements for the integration of the digital map processor. This effort is priced on a 
time and materials basis. Therefore any savings achieved from using the prototype method 
will be passed along to the Government in the form of reduced overall labor hours. 
However, a significant portion of the contract is firm-fixed-price, and any savings achieved 
through incorporation of prototyping on this contract line item will not pass to the 
Government. 

3. Software Process Maturity and Process Improvement 

The lack of a formal assessment of LORAL's software process maturity, and a 
resulting process improvement plan are significant issues for the Government. The 
maturity of LORAL's software processes in use on the SOA program have never been 
independently assessed by the Government. As result, the Government lacks a 
quantifiable standard, such as the SEI CMM by which to measure the maturity of 
LORAL's process. In conjunction with this, there does not appear to be a formal process 
improvement plan specifically addressing the SOA software processes. These are issues 
because the use of process assessments, and the process improvement plans which are 
derived from them, are called for in policy [Ref. 13 p. 6-D-l-l] and have been proven to 
be effective in reducing the cost of software development and maintenance. 

For example, a recent article in Cross Talk, The Journal of Defense Software 
Engineering [Ref. 81, pp. 14-17] illustrated the economic benefits of software process 
improvement. The article discusses a study which was performed by Software 
Productivity Research Incorporated in the test software branches of the Air Force's 
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Oklahoma Air Logistics Center, Software Division. The purpose of the study, 
commissioned by the Air Force's software executive, was to quantify the economic 
benefits of investing in software process improvement. Four projects, which spanned from 
1986 to 1995, were included in the study. The study concluded that a process 
improvement investment of $1.5 million over an eight year period, yielded a cost savings 
of $11.3 million, a return on investment of 7.5 to 1. The benefits were achieved in these 
areas: 


• 90 percent reduction in defect rates. 

• 26 percent reduction in the average cost of a maintenance action over the last 
two years. 

• Productivity increased to levels ten times that of the baseline project. 

This article clearly demonstrates the economic benefits of process assessments and 
of investing in process improvement. 

E. LESSONS LEARNED 

As a result of the analysis of the SOA PDSS case, the following lessons have been 
learned: 

• The cost of software development, whether new development or PDSS is a 
function of factors influenced by both the Government and the contractor. As 
such, costs must be controlled through actions on the part of both the 
Government and the contractor. 

• Documentation is expensive. The type and amount of documentation purchased 
by the Government should reflect what is needed to support the future strategy 
and needs of the program. 

• Even in the absence of a formal acquisition strategy, the pattern of actions and 
decisions made in a software-intensive program can indicate the selection of a 
certain strategy. Decisions made throughout a development program have 
"downstream" impacts. 

• The CRLCMP is the key management plan during the operations and 
maintenance phase of the software lifecycle. It should be maintained and 
updated to reflect the current and future PDSS concept. 
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• System specific knowledge and expertise are critical in maintaining 
software-intensive systems. 

• Transition from one source to another for software maintenance is risky. It 
requires preparation and planning. 

• The PPBS process requires that PDSS requirements be planned at least two 
years in advance. Failure to properly forecast requirements leaves a program 
with inadequate fUnds to implement desired changes. 

• The development and maintenance of software-intensive systems creates a 
situation in which the Government and contractor are dependent on one another. 
The degree of cooperation in the relationship should reflect this mutual 
dependence. 

• The use of a firm-fixed-price contract for complex weapon systems software 
development and maintenance is inappropriate. Weapon systems software 
development or maintenance requires the use of contracts which adequately 
share cost risk, provide cost visibility, and provide incentives for the contractor 
to control cost. 

• The use of the SLOC metric is inaccurate for effort estimation. 

• The award fee incentive, whether used in a cost-reimbursement or fixed-price 
contract, provides a flexible and effective tool for motivating contractor 
performance. It also serves to improve communication between the 
Government and the contractor. 

• Proper management of software acquisition requires resources. These resources 
include systems and software engineering personnel, and software cost 
estimation tools. 

• Metrics provide the PM with valuable insight into the products and processes 
involved with software development. Lack of a software metrics program limits 
the PM's insight into the contractor's software process. 

• The age and capabilities of a program's software development tools contribute to 
some degree to the cost of the development or maintenance effort. Modem 
CASE tools can reduce software engineering labor costs through automation of 
functions such as configuration management and testing. 

• To be a true cost reduction benefit to the Government, use of the prototyping 
development methodology must be linked with a contract type which shares 
savings with the Government. 

• Software process assessments such as the SEI CMM provide the Government 
with insight into the maturity of a contractor's software process. These 
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assessments in turn provide the basis for process improvement plans. 
Investments made in process improvements yield future cost savings. 

F. CHAPTER SUMMARY 

This chapter has provided the identification and the analysis of the significant 
issues impacting the management of the SO A PDSS software acquisition. This analysis 
has revealed that the significant management issues with regard to the SOA PDSS are in 
the areas of acquisition management, project management, and software engineering. 
These issues are significant in that failure to address them hinders the PM in his efforts to 
become a "smart buyer", and to properly manage the PDSS program. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. INTRODUCTION 

The primary purpose of this chapter is to provide the conclusions and 
recommendations for this investigation. It consists of four major sections. The first 
section presents the conclusions drawn from the analysis of the issues identified in the 
SOA PDSS case, and the lessons learned. The second section consists of specific 
recommendations to address the issues. The third section discusses the answers to the 
research questions. Finally, areas for further research are provided. 

B. CONCLUSIONS 

The following are the conclusions drawn from the analysis and lessons learned. 

1. General 

This thesis confirms the findings of the GAO [Ref. 2], the Defense Science Board 
[Ref. 10], and author Robert L. Glass [Ref. 9], These and numerous other reports and 
articles concerning DoD software have emphasized that the key to success in weapon 
system software acquisition is effective management by PMs. Effective software 
management requires the use of highly skilled personnel resources, tools, and methods. 
Perhaps the most essential element of effective program management is the support of 
senior leaders who are well informed in the complexities of software development. These 
leaders must understand that software management is vastly more difficult than hardware 
management. The reasons for this difficulty, presented in Chapter II include: 

• The lack of adequate software management concepts, methods, and practices. 

• The complexity of mission critical software systems. 

• The invisibility of software. It is difficult to measure its essential characteristics 
and those of the software process. 

• The difficulty in defining software requirements. 

• Software is changeable, flexible and modifiable. Because it has these qualities, 
control of the modification process is integral to the effectiveness of software. 
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In the absence of effective software management, these difficulties often dominate 
software acquisition. Especially important is management of software systems as they 
evolve during PDSS. Without the proper resources mentioned previously to support 
effective management of PDSS, the software which now controls modem weapon systems 
may not be ready when called upon to perform its mission. 

2. Acquisition Management 

Analysis of the acquisition management issues has revealed the following: First, 
that the Government sees cost as the underlying issue in the SO A PDSS case, with their 
perception being that LORAL is too costly. However, the researcher concludes that the 
tme issue in this regard is not the magnitude of the cost, but rather how the cost is 
managed by both the Government and LORAL. Clearly, the Government needs to do 
more to manage the cost it believes is excessive than simply awarding a firm-fixed-price 
contract which does nothing more than place an upper limit on cost. If the PDSS cost is 
genuinely a Government concern, its elements should be analyzed in detail and strategies 
to reduce it must be developed jointly with LORAL. In addition, because cost is such an 
important issue from the Government's perspective, it is in LORAL's interest to take 
sincere actions to evaluate ways to reduce it. Because the major costs are directly 
associated with manpower, this will inevitably require LORAL to reduce staffing on the 
project. Failure to find such economies, may result in the loss of the SOA business. 

Second, program plans are essential to the effective management of the program. 
Currently the only planning mechanism in use on the SOA PDSS program is the contract 
SOW. Without the framework which a program plan provides, management is ad hoc, an 
exercise in crisis management, not program management. The most logical planning 
mechanism to use for PDSS is the SOA CRMP. Lack of a current CRMP leaves the PM 
without the framework necessary to plan the future of the PDSS program, and it fosters a 
sort-term perspective. This short-term perspective is evident in two primary areas. First, 
is the mistaken belief in the user community that a competitive procurement of PDSS 
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following this contract is a genuine possibility. With regard to this issue, the researcher 
concludes that because system knowledge and experience are so critical in maintaining 
complex, integrated software systems, LORAL is realistically the only source capable of 
performing PDSS. Second, the program lacks a computer resources master schedule 
which links planned IAS enhancements to the budget process. While SIMO is budgeting 
for enhancements as it prepares budget estimates, there is not a single integrated plan 
which documents this. Lack of such a plan leaves the PM without visibility into the future 
requirements and a budget to implement those requirements. In the first two years of the 
PDSS program, lack of such a plan left the PM with budgets which appeared to be tight in 
relation to the cost to implement the desired enhancements. 

Third, the Government and LORAL are dependent on one another in this 
relationship. The level of interdependence in the buyer-seller relationship clearly dictates a 
closer, more cooperative relationship. Fourth, the use of a firm-fixed-price contract does 
not support the PM's goal of becoming a smart buyer. It limits the Government's visibility 
of the actual cost of PDSS, and provides no direct incentive to LORAL to reduce the 
overall contract cost. The award fee incentive is perhaps one way to assist the PM in 
achieving his goal. It is a flexible motivation tool and it enhances communication between 
the parties. 

3. Project Management 

The analysis of the SOA PDSS case revealed that the two significant issues in 
regard to project management are project management resources and software metrics. 
Analysis of these issues revealed the following. First, management of weapon systems 
software acquisition requires specific resources. The PM currently lacks the resources 
needed to properly manage the complexities of software acquisition. These resources 
include systems and software engineering personnel with system knowledge, and 
automated tools to estimate software costs. Even if there was to be a change in PDSS 
contractors in an effort to find a lower price, the PM is still not properly equipped to 
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manage them. Second, management of software acquisition also requires metrics. 
Without the use of software metrics in the SOA PDSS program, the PM is blind to the 
essential details of LORAL's software process, quality, and productivity. 

4. Software Engineering 

The area of software engineering presents three significant management issues. 
These issues are: 

• The age of the software development tools and language, 

• Requirements definition and development methodology, and 

• Software process maturity and process improvement. 

Analysis of these issues revealed first that the age of the software tools and the 
JOVIAL language, for which CASE technology is not available, is contributing to some 
degree to the perceived high cost of PDSS. Second, while LORAL wants to expand the 
use of prototyping to evolve exact software requirements, any savings achieved through 
prototyping would not be shared with the Government under the current firm-fixed-price 
contract. Finally, the Government has never formally assessed the maturity of LORAL's 
software processes. Without a formal assessment of LORAL's processes, the maturity of 
those processes in relation to a known standard such as the SEI's CMM, is unknown. 
Also, without a formal process assessment, there is no basis for a detailed process 
improvement plan. 

C. RECOMMENDATIONS 

Based on the analysis in Chapter IV of this thesis, and the conclusions of the 
previous paragraphs, the following recommendations are provided. These 
recommendations seek to improve the management of the SOA PDSS program. 

1. Provide the PM with the necessary software management resources. 

If TAPO is to continue management of the PDSS program, it must be equipped 
with the resources to do so. These resources should include a systems engineer, a 
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software engineer, and the automated project management and cost estimation tools 
required to perform basic management functions. The process of gathering these 
resources should begin by filling TAPO's now vacant software engineer position. These 
resources will support the PM*s capability to perform project management functions which 
mirror those of LORAL. These functions include assessment of the system impact of 
proposed enhancements or corrections, cost estimation of enhancements and corrections, 
analysis of metrics data, and configuration management. Without these minimum 
management resources, it is doubtful that the cost of PDSS will ever be properly managed, 
regardless of the contractor. An annual contract of approximately $4 million requires 
Government management oversight. 

2. Develop a more cooperative relationship with LORAL 

The pattern of actions taken in regard to PDSS indicates a strategy which includes 
LORAL as the SOA systems integrator. As stated previously, LORAL is currently the 
only source that can realistically perform PDSS on the complex SOA system. This 
strategy should be formalized in the SOA CRMP and form the basis of a closer, more 
cooperative relationship with LORAL for a second three year contract. One of the keys 
to a more cooperative relationship will be a contract type which is fair and meets the 
objectives of both parties. One of LORAL's objectives in particular is to maintain stability 
in the SOA workforce - to have a predictable backlog. This was in essence the purpose of 
the "framework" established by the base effort of the first option year. One of the 
Government's objectives for the contract is the ability to put new requirements (such as 
digital map integration) into the contract effort at any time. In addition to these particular 
objectives, the new contract should provide the following: a more equitable sharing of 
cost risk. Government visibility as to the true costs of PDSS, a positive incentive for 
LORAL to reduce costs, and, a vehicle for more open communication. 

One particular contract form which may assist the parties in meeting mutual goals 
is the task order contract. While not one of the particular contract types discussed in 
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Chapter IV, it may include provisions for awarding task orders priced in several different 
ways. A task order contract is defined as: 

...a contract for services that does not procure or specify a firm 
quantity of services (other than a minimum or maximum quantity) and that 
provides for the issuance of orders for the performance of tasks during the 
period of the contract. [Ref. 82, p. 12] 

The task order contract should be structured with a minimum and maximum on 
engineering labor hours. The minimum should be awarded annually on a CPAF basis to 
maintain a minimum level of staffing working on SOA software issues for the entire year 
of support. This would provide LORAL the consistency it needs in planning project 
staffing. It also provides the Government with additional insight and control it needs. 
Additionally, the increased communication required by the award fee evaluation process 
will be key in development of a more cooperative relationship. The suggested award fee 
criteria would address the following areas: 

• Cost control. 

• Program Management. 

• Technical. 

° Software process improvement. 

° Software testing and quality. 

° Software metrics program. 

As the Government generates additional requirements during the period of 
performance, additional tasks may be awarded. These can be priced by the most 
appropriate means for the task, in accordance with the contract types specified in the 
terms of the contract. Simple tasks which are low risk and easily priced may be 
firm-fixed-price. Complex integration efforts involving higher risk may be priced on a 
CPAF basis. 

The use of a task order contract will facilitate a cooperative relationship because 
the award of an annual minimum effort will show a commitment to LORAL on the 
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Government's part. It will also provide the Government the flexibility to meet new 
requirements as they become known. One particular task the Government should consider 
awarding is a study to further define what would be required to transition to another 
source (either a Government SSA or another contractor) in the future. The Government 
may ultimately decide to stay with LORAL for the life of the program. However, the PM 
should continue to evaluate future PDSS options. A generic listing of transition 
considerations is provided in Appendix B of this thesis. 

3. Reestablish a system of program planning through the SOA CRMP 

The program's planning needs can be fulfilled in a single plan, the SOA CRMP. 
The PM should initiate actions to update CRMP to reflect the new PDSS concept. 
Formalizing the strategy in the CRMP will serve to solidify a commitment to LORAL and 
provide the much needed long-range direction for the program. Special attention should 
be paid to the development of a computer resources master schedule which forecasts 
requirements a minimum of two years into the future. The concept behind this schedule, 
depicted graphically in Figure 20, is to establish a link between planned requirements, cost 
estimates of the implementation of those requirements, and the SOA PDSS budget. 
Planning to this level of detail requires the PM to have the resources discussed earlier. 
Without a planning process such as this, reflected in a schedule, PDSS will always appear 
to be expensive, because the budget will not support system requirements. A schedule of 
this nature will also serve to provide visibility to both the PM and the user of the cost of 
PDSS. It will assist them in prioritizing requirements and in making the tradeoffs required 
when budgets are relatively fixed and thus a constraining factor. 
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Figure 20. Computer Resources Master Schedule 

4. Initiate a software metrics program 

One of the keys to meeting the PM's goal of becoming a "smart buyer" is to 
increase insight into LORAL's software development process. Perhaps the most effective 
way to increase insight is to gather and analyze pertinent metrics data. To be a truly 
useful benefit, which will warrant the additional effort and expense involved in gathering 
and analyzing the data, the selected metrics must address issues which concern both the 
PM and the user. A particularly useful method to define the metrics set is the 
Goal/Question/Metric paradigm created by Victor Basili. [Ref. 83] This method has been 
effectively applied in the metrics program at Hewlett-Packard. [Refs. 21, 84] 

The principle behind the paradigm is that each software project has a set of goals. 
For each goal, there is a set of questions that may be asked to determine if those goals are 
being achieved. Many of these questions can be answered with measurable metrics data. 
The relationship between the goals, questions, and metrics are shown in Figure 21. 
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Goal 1 


Goal 2 



Metric 1 Metric 2 Metric 3 Metric 4 Metric 5 


Figure 21. The Goal/Question/Metric Paradigm 
From [Ref. 21] 

Using this method, one of the goals of the Hewlett-Packard metrics program is to 
minimize engineering effort. In the SOA PDSS case, since the Government feels that 
PDSS costs too much with LORAL, minimizing engineering effort may also be one of 
their goals. Several of the questions and corresponding metrics which Hewlett-Packard 
uses to achieve this goal are: 

• Question: Where are the resources going? Where are the worst rework loops in 

the process? 

° Metric: Engineering months by product/product component/process activity. 

• Question: How much do the maintenance phase activities cost? 

° Metric: Engineering time and cost. 

• Question: What are the major cost components? What aspects affect the cost? 

° Metric: Engineering months by product/product component/process activity. 

• Question: How do costs change over time? 

° Metric: Track cost components over the entire maintenance lifecycle. 
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• Question. How maintainable is the software product as changes occur? When 

do I give up and rewrite? 

° Metrics: Incoming problem rate. 

° Defect density. 

° Code stability. 

° Code complexity. 

° Number of modules changed to fix one defect. 

• Question: What will the maintenance requirements be? 

° Metrics: Code stability, complexity, size. 

° Prerelease defect density. 

The exact goals, questions, and metrics are issues which need to be defined jointly 
by the user, the PM, and LORAL. The point is that to be a "smart buyer" and to begin 
making progress toward program goals, the program needs a metrics program. The 
researcher recommends beginning a metrics program gradually with a single metric. This 
metric should track estimated versus actual engineering effort by product and by phase of 
the engineering process. This metric will serve two purposes. First it will provide insight 
as to where development resources are going. Second it will assist the Government in 
developing insight as to the accuracy of LORAL's cost estimation system. In further 
defining a metrics program, the PM should make maximum use of LORAL's current 
metrics system. Other sources to consult in developing a metrics program include the 
Army's Software Test and Evaluation Panel (STEP) metrics program, and the Air Force's 
Guidelines for Successful Acquisition and Management of Software Intensive Systems 
[Ref. 20] 

5. Conduct a formal software capability evaluation of LORAL's SOA 

process 

The researcher recommends that the PM conduct a formal assessment of the 
maturity of LORAL's SOA processes. A formal assessment will serve two purposes. First 
it will provide the Government with valuable insight into the details of LORAL's software 
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development process. This benefit would be maximized if the assessment could be 
conducted by trained Government software engineering personnel. If not, the PM should 
consider the use of a consulting firm such as Software Productivity Research to conduct 
such an assessment. Second, the assessment will serve as the basis for the development of 
a process improvement plan which will seek to maximize the performance of the SOA 
software process. This may also require the Government to share in the cost of 
implementing such a plan. 

D. ANSWERS TO RESEARCH QUESTIONS 

This section provides summarized answers to the questions which guided this 
research. 

1. Primary Research Question 

The primary research question for this thesis is: What are the significant 
management issues associated with the current SOA PDSS program and what 
actions can be taken to address these issues? 

Analysis of the SOA PDSS case revealed that the significant management issues 
are found in the areas of acquisition management, project management, and software 
engineering. These issues can be addressed through actions which require the 
Government to better manage software acquisition. The details of such actions were 
previously discussed. 

2. Subsidiary Research Questions 

a. What is PDSS and what are its essential elements? 

PDSS is the process of ensuring that fielded weapon system software 
continues to meet the needs of the user. It includes two categories of software 
"maintenance" activities. These categories are corrective maintenance, activities focused 
on the correction of software defects; and enhancements, activities performed to either 
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accommodate changes in the operational environment or to improve the software's 
performance. PDSS requires detailed planning prior to deployment of the software 
system, and a structured software engineering approach similar to that required during 
software development. 

b. What are the principal policies and guidance concerning 

PDSS? 

The principal policy concerning PDSS of Army weapon system software is 
provided by the draft policy on PDSS. This policy seeks to improve the planning and 
preparation for PDSS, and to centralize at the DA level the funding and priorities for 
PDSS. The only DoD guidance concerning how to conduct PDSS is given by Military 
Handbook 347, Mission-Critical Computer Resources. This handbook addresses both the 
preparation for PDSS, and the conduct of PDSS. 

c. What are the elements of the current SOA PDSS program? 

The SOA PDSS program consists of the key players, interacting in the 
three processes involved in software acquisition. The key players are the user, the 160th 
Special Operations Aviation Regiment (Airborne); the PM, a project officer at the 
Technology Applications Program Office; and the contractor, LORAL Federal Systems 
Company of Owego, New York. In the acquisition of PDSS for the SOA aircraft, these 
players interact in three processes. These processes are acquisition management; project 
management; and software engineering. 

d. What are the significant management issues associated with 

the current PDSS program? 

Analysis of the SOA PDSS case revealed that the significant management 
issues are the following: 

* Acquisition Management Issues 

° Cost 

° Program Plans 
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° The Buyer-Seller Relationship 

° Contract Type 

Project Management Issues 

° Project Management Resources 

° Software Metrics 

Software Engineering Issues 

° Software Development Toolset and Language 

° Requirements Definition and Development Methodology 

° Software Process Maturity and Process Improvement 

e. What actions can be taken to address these issues? 

In response to these issues, the researcher recommends the following 

The PM be provided with the necessary software management resources. These 
resources include systems and software engineering personnel, and automated 
tools to assist in developing software cost estimates. 

The PM should formalize a PDSS concept and strategy which includes a 
cooperative relationship with LORAL. 

The PM should reestablish a system of program planning through the SOA 
CRMP. 

The PM should initiate a software metrics program. 

The PM should conduct a formal software capability evaluation of LORAL's 
SOA process. 

This evaluation should lead to the development of a process improvement plan. 

f. What lessons can be learned from the SOA program’s 
experience with PDSS? 

The principal lessons learned from the SOA PDSS case are: 

The cost of PDSS is a function of factors on both sides of the buyer-seller 
relationship. As such it must be addressed through actions on both sides of the 
relationship. 




• PDSS requirements must be planned in advance so that the budget will support 
the purchase of the desired functionality. 

• The firm-fixed-price contract is not appropriate for PDSS of complex weapon 
system software. The selected contract type must adequately share risk, provide 
visibility as to the true cost of software maintenance, and provide incentives to 
reduce costs. 

• Effective management of software acquisition requires adequate resources. 
These resources include systems and software engineering personnel and 
software cost estimation tools. 

E. AREAS FOR FURTHER RESEARCH 

The following areas are recommended for further research: 

1. A Software Management Capability Model 

Development of tool and model to assess the software acquisition management 
capabilities of DoD program management organizations. This thesis confirmed the 
findings of the GAO [Ref. 2] and the Defense Science Board [Ref. 10], These reports 
concluded that despite the recent improvements in software engineering methods and 
technology, effective management is a major key to success in software acquisition. A 
tool of this nature would be used to assess the particular strengths and weaknesses of a 
program management organization's software management capability. It would address 
personnel qualifications, tools, and methods. The model would also serve as the basis for 
management process improvement plans. 

2. Implementation of the Army Policy on PDSS 

The Army's draft policy on PDSS seeks to centralize the priorities and binding for 
PDSS of software-intensive systems. This study would evaluate the impact of the policy 
on weapon systems readiness, and the ability of users to implement desired enhancements. 

3. Pre-Deployment Software Support of the RAH-66 COMANCHE 

A study of the pre-deployment software support preparations for the Army's 
RAH-66 COMANCHE helicopter program. The SOA program represents one of the 
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Army's first experiences with highly integrated avionics systems. The SOA's 380,000 
source lines of code are dwarfed in comparison to the COMANCHE'S estimated one 
million lines of code. This study would examine the COMANCHE'S pre-deployment 
software support activities. Specifically it would investigate the PDSS concept, and 
whether the PM is prepared to implement that concept. 

4. The Army’s Systems and Software Engineering Capabilities 

A study of the systems and software engineering capabilities of the Army. Army 
weapon systems continue to increase in complexity and software content. As they do, 
technical personnel with system specific expertise will be critical to the Army's ability to 
manage these programs effectively. This thesis has shown that this may be an area which 
impacts other Army programs. A study of this nature would survey the Army's current 
systems and software engineering capabilities. It would determine if the Army has 
sufficient technical personnel, and if not, how the situation could be addressed. 

5. SOA PDSS 

This thesis has examined the SOA PDSS case for only the base, and first option 
years of the first PDSS contract. A follow-on study would examine the SOA PDSS 
program to determine if lessons learned have been applied in later contracts. 

F. CONCLUDING REMARKS 

In software acquisition, there are no "silver bullets". No single method or action 
will address all of the issues involved in a software acquisition management situation as 
complex as the SOA PDSS. It is doubtful that merely changing the contract type, or 
beginning to gather metrics data, or even changing sources, alone would "solve" the SOA 
PDSS "problem." Rather, solving the problem requires a management commitment to 
trying a combination of these methods, and a commitment to change methods when 
required. The issue is whether we as DoD managers manage our software issues or let the 
issues manage us. 
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APPENDIX A. ACRONYM LIST 


ACAT 

Acquisition Category 

ACs 

Airframe Contractors 

AHASS 

Army SOA Helicopter Avionics System Simulator 

ANVIS 

Aviator's Night Vision Imaging System 

ASE 

Aircraft Survivability Equipment 

ATCOM 

U.S. Army Aviation and Troop Command 

AVSCOM 

U.S. Army Aviation Systems Command 

BSAS 

Boeing-Sikorsky Air Services 

CASE 

Computer Aided Software Engineering 

CCB 

Configuration Control Board 

CDB 

Common Database 

CDU 

Control Display Unit 

CECOM 

U.S. Army Communications-Electronics Command 

CMFD 

Color Multifunction Display 

CMM 

Capability Maturity Model 

CMS 

Combat Mission Simulator 

CPAF 

Cost-Plus-Award-Fee 

CPFF 

Cost-Plus-Fixed-Fee 

CPIF 

Cost-Plus-Incentive-Fee 

CRLCMP 

Computer Resources Life-Cycle Management Plan 
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CRMP 

Computer Resources Management Plan 

CSC 

Computer Software Component 

CSCI 

Computer Software Configuration Item 

CSU 

Computer Software Unit 

DA 

Department of the Army 

DCE 

Decompression Engine 

DCSOPS 

Deputy Chief of Staff Operations 

DLSIE 

Defense Logistics Studies Information Exchange 

DMP 

Display Management Program 

DoD 

Department of Defense 

DP 

Display Processor 

DPML 

Display Processor Memory Loader 

DTIC 

Defense Technological Information Center 

DTS 

Data Transfer System 

DTT 

Desk Top Trainer 

ECP 

Engineering Change Proposal 

EUT&E 

Early User Test and Evaluation 

FAR 

Federal Acquisition Regulation 

FFP 

Firm-Fixed-Price 

FFP, LOE 

Firm-Fixed-Price, Level of Effort Term 

FLIR 

Forward Looking Infrared 
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FPE 

Fixed-Price with Economic Price Adjustment 

FPIF 

Fixed-Price Incentive, Firm Target 

FPIS 

Fixed-Price Incentive, Successive Target 

FPRP 

Fixed-Price with Prospective Price Redetermination 

FPRR 

Fixed-Ceiling-Price with Retroactive Price Redetermination 

FRR 

Flight Readiness Review 

GAO 

General Accounting Office 

GPS 

Global Positioning System 

GSS SLD 

Ground Support System Software Load Device 

GTR 

Government Trouble Report 

HF 

High Frequency 

HOL 

Higher Order Language 

LAP 

Integrated Avionics Program 

IAS 

Integrated Avionics Subsystem 

IEEE 

Institute of Electrical and Electronics Engineers 

ILS 

Integrated Logistics Support 

INU 

Inertial Navigation Unit 

IOT&E 

Initial Operational Test and Evaluation 

IPR 

In-Process Review 

rv&v 

Independent Verification and Validation 

LCCS 

Life Cycle Contractor Support 
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LCSE 

Lifecycle Software Engineering 

LOC 

Lines of Code 

LRIP 

Low Rate Initial Production 

LRU 

Line Replaceable Unit 

MCCR 

Mission Critical Computer Resources 

MDG 

Map Display Generator 

MDGML 

Map Display Generator Memory Loader 

MEP 

Mission Equipment Package 

MFP 

Major Force Program 

MICOM 

U.S. Army Missile Command 

MMFD 

Monochromatic Multifunction Display 

MMP 

Map Management Processor 

MMR 

Multimode Radar 

MMS 

Mission Management System 

MP 

Mission Processor 

MY 

Man Year 

NASA 

National Aeronautics and Space Administration 

NDI 

Non-developmental Item 

ODISC4 

Office of the Director Information Systems for Command, Control, 

Communications, and Computers 

OFP 

Operational Flight Program 
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O&M 

Operations and Maintenance 

PAT 

Process Action Team 

PBIT 

Power-On-Built-In-T est 

PDSS 

Post Deployment Software Support 

PEO 

Program Executive Officer 

pros 

Prime Item Development Specification 

PM 

Program Manager 

PMSOA 

Product Manager, Special Operations Aircraft 

PPBS 

Planning, Programming, and Budgeting System 

PTT 

Part Task Trainer 

RDT&E 

Research, Development, Test and Evaluation 

ROC 

Required Operational Capability 

RTU 

Remote Terminal Unit 

RTUML 

Remote Terminal Unit Memory Loader 

SATCOM 

Satellite Communication 

SEE 

Software Engineering Environment 

SEI 

Software Engineering Institute 

SGP 

Signal Generator Program 

SIMO 

Systems Integration Management Office 

STNCGARS 

Single Channel Ground and Airborne Radio Subsystem 

SLOC 

Source Lines of Code 
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SOA 

Special Operations Aircraft 

SOF 

Special Operations Forces 

SOF MOD 

Special Operations Forces Helicopter Modification 

SOW 

Statement of Work 

SROMP 

Start-Up Read Only Memory Program 

SRS 

Software Requirements Specification 

SSA 

Software Support Activity 

STR 

Software Trouble Report 

STRICOM 

Simulation, Training, and Instrumentation Command 

TAPO 

Technology Applications Program Office 

TF 

Terrain Following 

TIM 

Technical Interchange Meetings 

QA 

Quality Assurance 
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APPENDIX B. SOFTWARE TRANSITION CONSIDERATIONS 


A. PURPOSE 

This appendix describes the factors which must be considered in the transition of 
software support from one activity to another. Much of the literature concerning software 
support transition assumes that transition occurs between a commercial developer and a 
Government software support activity (SSA). However, similar procedures would be 
required in the orderly conduct of transition from one commercial contractor to another. 

B. DEFINITION 

Military Handbook 347, Mission-Critical Computer Resources Software Support, 
defines software transition as: [Ref. 25, p. 10] 

A controlled and coordinated sequence of actions wherein 
software development passes from the organization performing initial 
software development to the organization performing PDSS. Software 
transition also involves the SSA moving from pre- to post-deployment 
software support activities. 

The researcher has emphasized controlled and coordinated because the capability 
of the new support activity (whether Government or contractor) to properly support the 
software depends on what occurs or fails to occur during transition. Transition should not 
be conducted hastily. In fact software maintenance expert Thomas M. Pigoski believes 
that a good transition should take place over a two to three year period. [Ref. 31, p. 47] 
Implied in this definition is the fact that during transition, the PM will be managing the 
coordinated activities of two software organizations. Also implied are the increased costs 
the Government will bear while paying for the activities of two organizations during the 
transition. 

C. TRANSITION CONSIDERATIONS 

The following sections discuss the major considerations to be addressed during the 
transition of software support from one organization to another. 
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1. Transfer of System Knowledge 

Because system specific knowledge is so critical in software development and 
maintenance, this is perhaps the most important consideration in software transition. The 
system specific expertise resident in the developer must be transferred to the support 
organization during transition. The considerations discussed in the sections below support 
the transfer of system knowledge in a controlled and coordinated manner. 

2. Management and Administration Procedures 

The goal of these procedures is to ensure that the necessary management structure 
is in place to control the transition. This includes a memorandum of agreement between 
the developer and the SSA which outlines specific responsibilities. If the transition is to 
occur between two contractors, an acquisition plan which defines the specific contracting 
approach will be critical to a coordinated transition from one contractor to another. 
Included in this consideration is the update and transfer of program plans such as the 
project management plan, configuration management plan, software development plan, 
and quality and test plans. Together, these plans will define for the new support activity 
how the system is supported and what tools are required to maintain the software. 

3. Software Engineering and Test Procedures 

This consideration addresses the software engineering environment and the 
development procedures used in maintaining the software. For a program such as SOA, 
which was developed on software tools which are 1980s technology, transition may be the 
logical point at which to introduce more modem tools. According to researcher Dr. 
Thomas Vollman, specific consideration should be given to tools which can accomplish 
the following: [Ref. 29, p. 194] 

• Provide management insight into development progress. 

• Provide quantitative metrics assessments of aspects of the software system as 
designed and built. 

• Define test requirements for upgrades based on specification changes. 


118 



• Provide measures of how well the software has been tested. 

• Provide for automation of configuration management activities. These include 
code management and control, and software builds and version descriptions. 

• Provide support for engineering changes to the software, including describing 
the structure of the code, and documentation of those structures. 

4. Training and Personnel 

Perhaps the most critical consideration in the transfer of knowledge during 
transition is the training of the software maintenance personnel. As discussed previously, 
system specific knowledge is key in maintaining software systems. This knowledge 
transfer will take time. It must be accomplished gradually, from one activity to another 
through a program of "hands-on," on-site training. Personnel from the organization 
assuming the support role must be intimately involved in the software development and 
maintenance process of the development organization as soon as possible. As the new 
organization's software support environment is established, personnel from the 
development organization should then be involved in training personnel at the new site. 
Pigoski calls these individuals "maintenance escorts." [Ref. 31, p. 48] This training 
culminates with the development, testing, delivery, and acceptance by the user of a new 
software release. 

5. Technical Documentation Procedures 

Large programs often have voluminous documentation. Often this documentation 
is out of date. The transition process should include a plan for the systematic update of 
system documentation and the transfer of the Technical Data Library from the developer 
to the maintainer. In addition to the plans already addressed, the key documents to 
consider are the following: 

• software requirements specifications 

• software design documents 

• software product specifications 

• version description documents 
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D. TRANSITION REFERENCES 


The following references are provided to assist the PM in further examination of 
the issues involved in transition of software support: 

• Department of Defense, MIL-HDBK-347, Military Handbook: Mission-Critical 
Computer Resources Software Support, 22 May 1990. 

• Pigoski, Thomas, M., Life Cycle Strategy: Software Support on the Front Line, 
Software Maintenance News, Inc., 1995. 

• Vollman, Thomas, "Transitioning From Development to Maintenance," 
Proceedings of the 1990 IEEE Conference on Software Maintenance, IEEE 
Computer Society Press, 1990. 
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